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PREFACE 

Although  the  Health  Standards  and  Quality  Bureau  (HSQB)  had  previously 
funded  several  special  projects  in  ambulatory  care  quality  assurance, 
the  Bureau  initiated  a  systematic  investigation  of  formal  ambulatory 
care  review  programs  in  1975.    Findings  of  this  investigation*  highlighted 
the  need  for  careful  examination  of  alternative  ambulatory  care  review 
approaches  to  determine  their  feasibility,  especially  in  the  PSRO 
environment. 

In  1976,  under  the  guidance  of  Michael  Goran,  M.D.,  former  Director, 
HSQB,  a  demonstration  program  was  initiated  to  further  examine  ambulatory 
care  review  methods  within  Professional  Standards  Review  Organizations. 

On  July  1,  1976  a  letter  from  HSQB  was  sent  to  all  the  PSROs  invit- 
ing them  to  participate  in  an  ambulatory  care  review  demonstration  pro- 
ject.   These  letters  specified  that  "the  demonstrations  will  be  designed 
to  assess  various  approaches  to  ambulatory  care  quality  assurance  in  a 
variety  of  health  service  delivery  systems  with  substantial  variety  in 
the  method  of  obtaining  a  rational  PSRO  coordination  approach  for  each 
project."    Five  demonstration  projects  were  selected  to  participate  in 
the  demonstration  project.    Representing  a  variety  of  locations  and 
proposed  methods  the  five  projects  began  formal  planning  activities  in 
early  October  1976.    The  demonstration  projects  are  Central  Massachusetts 
PSRO,  Bronx  PSRO,  Southcentral  Pennsylvania  PSRO,  Multnomah  Foundation 
for  Medical  Care,  and  New  Mexico  PSRO. 

At  the  same  time  as  the  demonstration  projects  were  being  selected 
an  analysis  contractor  was  selected  to  "assist  PSRO  in  developing  the 
demonstration  protocols  and  be  responsible  for  assessment  and  analysis 

♦Contract  No.  240-75-0101,  Bureau  of  Quality  Assurance,  Health  Services 
Administration 
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of  the  effectiveness  of  each  of  the  demonstration  projects  in  conjunction 
with  the  PSROs."    Health  Care  Management  Systems,  Inc.  (HCMS)  has  performed 
as  the  ambulatory  demonstration  analysis  contractor  since  July,  1976. 
HCMS  was  also  involved  in  a  three-year  grant  program  funded  by  The 
Robert  Wood  Johnson  Foundation  to  examine  ambulatory  quality  assurance 
programs  with  special  emphasis  on  information  systems.    Some  funding 
provided  by  the  Foundation  grant  was  used  to  give  a  private  perspective 
to  the  government-funded  demonstration  project  and  to  synthesize  results 
of  the  project  with  other  ambulatory  care  quality  assurance  research  and 
evaluation  programs  in  which  HCMS  was  engaged. 

This  report  on  the  development  of  ambulatory  care  review  methods  in 
the  five  PSROs  demonstration  projects  is  in  partial  fulfillment  of  the 
analysis  contract  requirements.    The  report  includes  two  volumes  and 
two  appendices: 

Program  Overview  and  Discussion 
Bronx  PSRO 

Central  Massachusetts  PSRO 
Multnomah  Foundation  for  Medical  Care 
New  Mexico  PSRO 

Southcentral  Pennsylvania  PSRO 
Policy  Recommendations  and  Technical  Guidelines 
Interim  Report  -  Cooperative  Demonstration  and  Analysis 
of  Ambulatory  Care  Quality  Assurance  Review  Methodology 
in  PSRO  Settings  -  November  1977 

Ambulatory  Medical  Care  Quality  Assurance  1977  Conference 
Proceedings,  La  Jolla  Health  Science  Publications,  La 
Jolla,  California  1977 
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Volume  I 
Section  1 
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5 

Volume  II 
Appendix  1 


Appendix 


Project  overviews  and  data  for  each  of  the  demonstration  projects 
are  documented  in  Volume  I.    This  volume  is  especially  useful  as  examples 
of  ambulatory  care  review  developmental  efforts  in  PSRO.    Volume  II, 
"Policy  Recommendations  and  Technical  Guidelines"  are  directed  to  HSQB  and  other 
agencies  involved  in  medical  care  quality  assurance  and  utilization 
review  in  a  regulatory  environment.    The  appendices  include  the  Interim 
Report  which  describes  the  first  year  demonstration  project  activity 
and  findings.    The  second  appendix  is  a  book  containing  papers  presented 
at  a  two-day  national  conference  on  ambulatory  care  quality  assurance. 
The  entire  report  is  designed  to  assist  organizations  searching  for 
feasible  ambulatory  care  review  alternatives. 
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Assessment  activities  conducted  in  each  of  the  five  Professional 
Standards  Review  Organization  (PSRO)  ambulatory  review  projects  were 
concentrated  in  several  areas:    a)  description  of  project  administration, 

b)  description  of  activities  in  the  areas  of  planning  and  administration, 
topic  selection,  criteria  development,  data  collection,  data  processing, 
review  and  decision  making,  feedback  and  intervention,  and  restudy,  and 

c)  description  of  resources  expended  to  accomplish  the  project's  goals. 

The  general  objectives  of  this  final  report  are  to: 

1.  Summarize  the  PSRO  activity  by  presenting  a  variety  of 
descriptive  documentation  (Volume  I). 

2.  Document  findings  of  investigation  of  review  techniques 
employed  by  each  PSRO  (Volume  I). 

3.  Guide  future  technical  assistance  conducted  by  HSQB  (Volume 
II). 

4.  Provide  recommendations  regarding  implementation  of  ambulatory 
care  review  systems  by  current  and  future  PSRO  demonstration 
projects  (Volume  II). 

Presented  in  this  volume  first  is  a  brief  program  overview  designed 
to  acquaint  the  reader  with  each  PSRO  project.    If  more  in-depth  informa- 
tion is  desired,  either  the  site  itself  or  HCMS  can  provide  a  more 
detailed  description.    The  next  three  sections  describe  activities  that 
occurred  at  each  project  and  the  results  of  these  activities.  Finally, 
the  last  section  presents  resource  data  for  each  PSRO  site. 

Of  the  five  PSROs  funded  in  the  demonstration  program,  four  are 
initiating  ambulatory  care  review  systems  where  no  prior  system  existed. 
They  are:    Central  Massachusetts  PSRO,  Bronx  PSRO,  Southcentral  Pennsylvania 
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PSRO,  and  Multnomah  Foundation  for  Medical  Care.    The  fifth  demonstration 
program,  New  Mexico  PSRO,  has  operated  a  claim-based  review  system  for 
several  years  under  contract  with  the  State  Medicaid  agency.    The  New 
Mexico  PSRO  demonstration  has  explored  some  specific  review  questions, 
most  of  which  were  research  oriented. 

HCMS  data  collection  for  this  report  has  continued  since  the 
beginning  of  the  demonstration  project.    Each  project  has  cooperated  in 
providing  many  different  types  of  data  including  questionnaire  informa- 
tion, counts  of  activities,  computer  run  sheets,  personnel  hourly 
activity  description,  minutes  of  meetings,  and  written  project  reports. 
The  cut-off  date  for  data  was  August  31,  1978,  except  for  resource  data 
which  was  discontinued  on  June  30,  1978.    Therefore,  the  most  recent 
activities  of  each  project  are  not  reflected  in  this  report.    The  reader 
should  be  aware  that  this  report  does  not  attempt  to  describe  the  entire 
data  base  of  each  project;  the  activities  and  data  described  are  meant 
to  be  representative  of  the  operations  of  each  demonstration  project. 
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PREFACE 

The  Multnomah  Foundation  for  Medical  Care  (MFMC)  was  awarded  a  two- 
year  contract  by  Health  Standards  and  Quality  Bureau  (HSQB)  to  test  a 
review  capability  which  applies  basic  principles  of  hospital  medical 
care  evaluations  (MCE)  used  by  Professional  Standards  Review  Organiza- 
tions (PSRO)  to  small  group  and  solo  practices.    MFMC  has  received  a 
six-month  extension  to  complete  the  project.    The  program  was  designed 
to  test  the  usefulness  of  a  physician  consensus  method  for  identifying 
topics  and  criteria  for  review,  based  on  perceived  problems  in  the  PSRO 
area,  and  to  examine  issues  concerning  the  most  effective  method  for 
data  collection  in  a  variety  of  practice  settings. 

MFMC  completed  two  cycles  of  data  collection  and  review  using  sets 
of  problem-oriented  topics  and  associated  criteria  sets.    The  scale  of 
the  project  was  relatively  small,  although  MFMC  sought  to  examine  topics 
which  were  believed  to  represent  potential  areawide  problems.    The  first 
cycle  of  review  involved  22  physicians  from  three  specialties;  the 
second  cycle  reviewed  48  physicians  from  five  specialties. 

A  second  part  of  the  demonstration  dealt  with  the  development  of  a 
modified  claim  data  base  simulated  claim  information  to  determine  the 
relative  effectiveness  of  using  different  increments  and  types  of  informa- 
tion for  screening  and  reviewing  medical  care.    The  objective  of  this 
part  of  the  demonstration  was  to  determine  the  minimum  claim  based  data 
set  necessary  to  identify  quality  related  problems  in  ambulatory  care. 
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Multnomah  Foundation  for  Medical  Care  submitted  an  initial  concept 
paper,  and  after  HSQB  modification  and  approval,  develop  preliminary 
and  final  plans  over  a  period  of  five  months. 

PROJECT  ADMINISTRATION 

The  Board  of  Directors  of  the  Multnomah  Foundation  for  Medical  Care 
developed  a  general  objective  for  the  demonstration  project  and  main- 
tained overall  responsibility  for  the  project.    The  Associate  Medical 
Director  of  the  Foundation  provided  overall  direction  for  the  project 
through  the  Ambulatory  Care  Review  Project  Management  Committee.  The 
latter  committee  was  composed  of  ten  practitioners  appointed  by  the  MFMC 
Board  of  Directors,  and  had  primary  responsibility  for  the  implementa- 
tion of  the  project.    Management  Committee  members  were  chosen  on  the 
basis  of  specialty  representation,  recognized  leadership  and  experience 
in  medical  care  evaluation.    The  activities  of  the  committee  included: 
identification  of  problem-oriented  topics,  development  of  criteria, 
development  of  the  data  collection  methodology,  selection  of  study 
participants,  review,  and  development  of  the  feedback  methodology.  The 
Management  Committee  also  provided  overall  project  guidance  and  made 
ongoing  refinements  in  project  design.    All  review  system  implementation 
plans  were  coordinated  by  the  Associate  Executive  Director  of  MFMC.  The 
organizational  structure  of  the  demonstration  is  illustrated  in  Figure  1. 

Two  other  committees  of  MFMC  provided  assistance  to  the  Management 
Committee.    The  PSRO  Criteria  Committee  assisted  in  criteria  development 
and  review,  and  the  Medical  Care  Evaluation  Committee  provided  review 
and  recommendations  regarding  the  project  progress  and  plans. 
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FIGURE  1 
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Since  technical  design  was  the  primary  responsibility  of  the 
Management  Committee,  staff  support  was  crucial  to  the  demonstration 
program.    Project  staff  included  an  associate  medical  director  who 
provided  overall  direction  for  the  project  and  chaired  the  Management 
Committee,  a  project  director  who  coordinated  management  and  technical 
implementation  of  the  program,  a  research  assistant  field  coordinator, 
two  PSRO  abstractors  and  seven  part  time  office-based  abstractors. 
Management  and  staff  reporting  lines  are  illustrated  in  Figure  1.  The 
project  staff  were  also  organized  to  utilize  the  expertise  of  MFMC  staff 
not  directly  related  to  the  demonstration  project.    A  staff  statistician 
was  employed  in  a  consulting  capacity  to  compile  and  reduce  data  for 
review  purposes. 

Financial  accounting  was  the  responsibility  of  the  Project  Director 
and  the  Executive  Director  of  MFMC.    The  Executive  Director  also  pro- 
vided ongoing  review  and  assistance  to  project  staff. 

TECHNICAL  COMPONENTS 

A.     Topic  Selection 

A  key  feature  of  the  MFMC  demonstration  program  was  testing  the 
usefulness  of  the  physician  consensus  technique  to  identify  potential 
problems  as  the  basis  for  review  topic  selection.    The  Management  Com- 
mittee, using  nominal  group  process  techniques,  followed  four  basic 
steps  in  topic  selection,  including: 

1.  Identification  of  possible  topics  for  review 

2.  Assigning  priorities  to  potential  topics 


3.  Evaluation  of  the  topics  according  to  criteria  for  appro- 
priateness 

4.  Consensus  decision  on  which  topics  to  proceed  with 

Use  of  information  sources  outside  the  Committee  itself  was  optional  for 
topic  selection  (e.g.,  literature,  consultants,  hospital  data,  etc.). 
An  essential  feature  of  the  process  was  having  the  physicians  identify 
based  on  their  experience  and  observations  problems  in  ambulatory  care. 
The  potential  problems  were  translated  into  perceived  problem  statements 
to  assist  in  focusing  criteria  development,  data  analysis,  review  and 
feedback  (see  Table  1:    MFMC  Review  Topics) . 

Topics  were  chosen  based  on  the  following  guidelines: 

1.  Perceived  deficit  in  quality  of  care 

2.  Frequent  occurrence  in  office  practice 

3.  Outcome  of  care  significantly  affected  by  one  or  more 
interventions 

4.  General  agreement  among  clinicians  on  the  importance  of 
intervention 

5.  Accessible  data 

The  first  cycle  of  review  included  six  topics.    Data  collection  for 
the  six  medical  care  evaluation  studies  was  conducted  simultaneously  in 
eight  practice  settings.    The  22  physician  participants  in  the  first 
round  of  review  included  internists,  family  practitioners  and  pediatricians. 

During  the  second  cycle  ten  new  topics  plus  four  of  the  topics 
from  the  first  review  cycle  were  reviewed.    The  same  three  specialties 
were  reviewed  with  the  addition  of  psychiatrists  and  obstetrics  and  gyne- 
cology.   HCMS  developed  Figure  2:    Data  Flow,  in  order  to  illustrate  the 
process  of  selecting  topics,  physicians  and  review  cases. 
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TABLE  1.    MFMC  REVIEW  TOPICS 


First  Cycle 


TOPIC 

1.  Management  of  hypertension* 

2.  Management  of  hyperactivity 
and  learning  disabilities* 

3.  Management  of  pediatric 
gastroenteritis 

4.  Completion  of  gonorrhea 
culture  when  indicated 

5.  Completion  of  pelvic  exam  with 
physical  exam* 


6.    Management  of  well  child  exam* 


PERCEIVED  PROBLEM 

Under-treatment  for  essential  hypertension 

Overuse  of  drugs  for  behavior  control 
without  adequate  workup  and  follow  up 

Inappropriate  use  of  contraindicated 
medications 

Failure  to  rule  out  gonorrhea  when  diagnoses 
are  made  that  are  associated  with  gonorrhea 

Failure  to  complete  a  pelvic  and  rectal 
exam  and  pap  smear  as  part  of  a  general 
physical  exam 

Differences  in  immunizations  and  exams 
between  family  practitioners  and  pediatri- 
cians 


Second  Cycle 


TOPIC 


PERCEIVED  PROBLEM 


Reactive  functional  hypoglycemia       Inappropriate  diagnosis 


Diabetes  mellitus  (insulin 
dependent) 


3.    Iron  deficiency  anemia 


4.    Chemistry  screen  results 


5.    Use  of  Coumadin 


6.  Lithium  and  treatment  of  bipolar 
depression  and  mania 

7.  Psychiatric  consultations 


Inappropriate  therapy  --  including 
patient  education 

Therapy  with  iron  alone  or  in  combina- 
tion without  minimal  study 

Inadequate  follow  up  for  abnormal 
chemistry  screen  results 

Anticoagulant  therapy  (coumadin)  without 
knowledge  of  coagulability  of  the  patient's 
blood,  i.e.,  treatment  without  monitoring 
prothrombin  time 

Treatment  without  management  of  blood 
levels 

Question  of  referral  source  not  answered 
by  the  consultant 


*Reviewed  also  during  the  second  review  cycle. 
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TABLE  1  (CONT'D) 

Second  Cycle 

TOPIC  PERCEIVED  PROBLEM 

8.  Abnormal  uterine  bleeding  Failure  to  diagnose  and  treat  abnormal 

uterine  bleeding 

9.  Fertility  Inappropriate  use  of  Laparoscopy 
10.    Colposcopy  Use  of  Colposcopy 
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FIGURE  2 
DATA  FLOW  IN  CASE  SELECTION 
MULTNOMAH  FOUNDATION  FOR  MEDICAL  CARE 
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Figure  2  Cont'd 
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B.      Criteria  Development 

Criteria  were  developed  by  the  Management  Committee,  again  using  a 
committee  consensus  technique.    Four  key  requirements  were  established 
for  potential  criteria: 

1.  Criteria  must  be  "essential."    (Essential  criteria  are  defined 

as  "those  elements  of  care  that  have  a  direct  impact  on  outcome*) 

2.  Criteria  must  be  relevant  to  the  perceived  problem 

3.  Recording  of  the  criterion  in  the  patient  record  should  be 
essential  to  good  quality  medical  care 

4.  Criteria  must  be  acceptable  to  the  medical  community 

MFfC  chose  to  establish  and  use  the  concept  of  "essential"  criteria 
based  on  the  need  to  collect  and  analyze  only  the  most  necessary  informa- 
tion for  determining  variations.    The  "essential"  criteria  were  the 
quality  of  care  parameters. 

During  the  first  round  of  review,  a  member  of  the  Project  Manage- 
ment Committee  was  assigned  to  each  topic  to  perform  background  study 
including  review  of  the  literature,  consultation  with  peers  and  drafting 
of  criteria.    The  Management  Committee,  using  the  reference  sources, 
further  developed  the  criteria  using  the  format  adapted  from  previous 
experience  of  the  American  Society  of  Internists. 

Criteria  lists  were  brief,  ranging  from  2  to  16  items  per  topic. 
A  complex  list  of  criteria  for  both  study  cycles  in  abbreviated  form  is 
found  in  Appendix  A.    Most  of  the  criteria  were  framed  as  statements 
(e.g.,  "if  no  resolution  in  six  weeks,  increase  dosage")  and  grouped  in 
a  number  of  general  categories  including:    history  and  physical  examina- 
tion, diagnostic  procedures,  therapy,  management,  contraindicated  therapy 

*MFMC  First  Year  Progress  Report,  September,  1977,  p.  4. 
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and  outcome.    Synonym  lists  and  drug  lists  accompanied  the  criteria 
where  relevant.    Each  criteria  set  specified  parameters  for  inclusion  of 
cases  in  the  study  by  patient  age  and  study  time  frame  (e.g.,  well  child 
exam:    patients  under  six  years  of  age  followed  for  at  least  three  well 
child  visits).    These  parameters  were  found  in  general  to  be  too  rigid 
in  the  first  cycle  of  review,  resulting  in  exclusion  of  approximately 
half  of  the  cases  identified.    Parameters  for  the  second  cycle  topics 
were  more  flexible.    In  the  second  cycle  of  review,  criteria  were 
developed  by  subcommittees  of  specialists.    They  were  later  ratified  in 
writing  by  all  participating  physicians. 

C.     Data  Collection  and  Processing 

During  the  planning  process  a  description  of  the  socioeconomic 
character  of  Multnomah  County  and  the  physician  population  was  compiled 
by  MFMC  staff  for  reference  in  planning  and  in  soliciting  physician 
participation.    Approximately  85  physicians  were  approached  to  partici- 
pate in  the  first  cycle  of  review,  of  whom  22  agreed.    During  the  second 
cycle  48  physicians  consented  to  participate.    Prior  to  data  collection 
each  participant  met  in  an  orientation  session  with  the  PSRO  staff  to 
discuss  project  procedures.    Preliminary  site  visits  were  carried  out  at 
each  practice  to  sign  participation  agreements,  review  record  systems 
and  perform  initial  chart  ratings. 

A  detailed  format  was  used  in  assessing  the  management  and  data 
flow  characteristics  of  each  office  during  the  preliminary  site  visits. 
Description  of  details  such  as  medical  record  filing  system,  types  of 
flow  sheet  forms,  reports  and  logs  used,  types  of  appointment  ledgers 
and  billing  processes  were  included  in  the  format.    An  approach  to  case 
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retrieval  and  abstracting  was  noted  for  each  of  the  study  topics  to  be 
reviewed  at  each  site  so  that  abstractor  training  could  be  suited  to  the 
particular  conventions  of  each  office.    Samples  of  encounter  forms  used 
in  each  practice  were  attached  to  the  protocol  and  filed  at  the  PSRO  for 
reference  purposes.    An  initial  chart  rating  was  performed  at  the  site 
visit  using  scaled  ratings  of  the  various  types  of  documentation  avail- 
able in  the  record.    A  sample  completed  office  profile  is  contained  in 
Appendix  B. 

Some  of  the  methods  for  identifying  and  locating  ambulatory  records 
(already  derived  from  the  office  profiles  for  implementation  in  data 
collection)  included: 

1.  Retrieval  through  an  automated  information  or  billing  system 

2.  Review  of  appointment  ledger  and  lab  log 

3.  Selection  of  charts  through  office  personnel  recollection  of 
patient  problem 

4.  Retrieved  data  via  bills  submitted  to  third  parties 

5.  Concurrent  logging  of  the  patient 

6.  Review  by  random  selection  of  medical  records 

The  case  selection  process  is  illustrated  in  Figure  2:    MFMC  Data  Flow, 
page  1:    Case  Selection.    The  office  profile  forms  also  proved  to  be 
useful  in  abstractor  training  and  in  initiating  abstracting. 

Abstracting  forms  and  instructions  were  developed  for  each  topic. 
Instructions  included  such  information  as  guidelines  for  charts  to 
pull,  narrative  description  of  procedures  with  examples  and  exceptions 
noted,  and  lists  of  drugs,  synonyms  and  concurrent  conditions. 
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The  data  collection  format  incorporated  the  use  of  numerical  codes, 
clinical  values,  plus  or  minus  signs  to  indicate  presence  or  absence, 
alphabetical  codes,  and  longhand  notes  on  the  abstracting  form. 

Pilot  testing  of  instructions  and  forms  was  conducted  at  the  first 
physician's  office  scheduled  for  abstracting.    Thereafter,  instructions 
were  rewritten  and  considerable  detail  was  added  in  all  areas,  including 
clearer  definitions  for  inclusion  of  cases  in  the  study.    The  nurse 
abstractor  and  a  nonclinically  oriented  abstractor  completed  a  one-week 
orientation  and  training  period.    In  addition  to  these  abstractors, 
seven  personnel  at  the  office  sites  were  assigned  to  complete  data 
collection  at  six  sites  and  were  given  a  brief  orientation  and  training 
session. 

As  the  data  collection  progressed,  abstractor  training  was  expanded 
to  include  medical  terminology,  training  in  medical  records  format, 
coding  systems,  use  and  interpretation  of  synonymous  data,  and  recogni- 
tion of  recording  errors. 

Abstractor  reliability  was  formally  tested  during  the  first  review 
cycle  using  three  levels  of  personnel.    The  mean  percent  of  accuracy  was 
82%  for  physicians,  83%  for  review  coordinators  and  87%  for  staff 
abstractors.*   As  a  standard  quality  control  measure,  20%  of  all  charts 
were  reabstracted  in  the  first  cycle  of  review.    In  the  second  cycle, 
each  chart  containing  a  variation  were  reviewed  for  accuracy  by  the 
field  coordinator. 

*MFMC  First  Year  Progress  Report,  Attachment  T:  Abstractor  Reliability 
and  Accuracy,  September,  1977,  p.  10. 
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A  sample  of  MFMC  abstracts  were  examined  (5%  of  the  total  abstracts) 
for  11  topics.    Frequency  counts  of  data  items  found  in  the  sample 
revealed  varying  degrees  of  data  availability  for  certain  categories  of 
data.    Diagnosis  and  therapy  categories  were  above  average  in  availability, 
history  and  physical  were  generally  near  the  average,  management  and 
outcome  categories  showed  low  availability  and  no  data  was  found  in  the 
sample  for  the  contraindicated  therapy  category.    These  findings  are 
described  in  more  detail  later  in  this  report. 

D.     Review,  Decision  Making  and  Feedback 

Review  involved  manual  scoring  of  the  abstract  and  summarization  of 
the  results.    Scoring  consisted  of  a  comparison  of  the  abstract  data  to 
the  criteria  and  a  determination  of  compliance  or  noncompliance  for  each 
criteria  item.    No  attempt  was  made  to  weight  the  results.    The  Manage- 
ment Committee  reviewed  initial  scoring,  revised  and  clarified  the 
reporting  format  and  stipulated  that  four  charts  per  physician  be  the 
minimum  number  abstracted  for  review. 

Variations  in  the  abstract  were  identified  by  the  abstractor  who 
then  reabstracted  the  chart  and  wrote  comments  interpreting  the  varia- 
tions.   The  variation  results  were  referred  to  the  Management  Committee 
for  review  and  determination  of  feedback  requirements.    The  process  of 
variation  identification,  review,  and  provider  feedback  is  illustrated 
in  Figure  2:    MFMC  Data  Flow,  page  2:    Abstracting  Review  and  Feedback. 

MFMC  staff  prepared  for  analysis  a  display  of  variation  data  by 
topic  and  physician  for  the  second  cycle  of  review.    A  chart  was  labeled 
as  a  "variation"  if  there  was  noncompliance  for  one  or  more  criteria. 
These  data  are  presented  later  in  this  report. 
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E.  Intervention 

MFMC  used  a  feedback  strategy  which  consisted  of  providing  results 
(variations)  to  practices,  then  discussing  the  variations  in  an  effort 
to  identify  potential  justifications.    Representative  records  with  prob- 
lems were  reviewed  with  the  practice  physicians.    No  comparative  or 
aggregate  data  were  presented.    Problems  were  generally  found  to  be  due 
either  to  documentation  problems  or  abstracting  errors,  or  were  due  to 
objections  to  the  criteria,  in  which  case  they  were  termed  "acceptable 
variations."    When  the  physician  was  not  accessible  or  refused  to  take 
part  in  the  feedback  meeting,  feedback  letters  were  sent.    A  sample  set 
of  variations  for  various  topics  is  found  in  Appendix  C. 

When  all  study  results  are  compiled,  MFMC  plans  to  send  displays  of 
comparative  data  to  physician  participants,  showing  performance  of  peers 
in  the  community  relative  to  the  physician's  own  performance. 

MFMC  plans  to  restudy  all  physicians  with  practice  variations. 
Restudy  will  consist  of  abstracting  and  reviewing  a  maximum  of  20  cases 
per  physician.    Logging  of  new  cases  is  now  under  way  in  most  of  the 
practices. 
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SECTION  III 
ANALYSIS  OF  DESIGN  AND  PLANNING 
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INTRODUCTION 

In  an  effort  to  identify  the  requirements  essential  for  developing 
a  Professional  Standards  Review  Organization  (PSRO)  ambulatory  care 
quality  assurance  capability,  Health  Care  Management  Systems,  Inc.  (HCMS) 
developed  and  administered  two  questionnaires  to  PSRO  ambulatory  care 
review  system  personnel  participating  in  the  demonstration  program.  The 
information  gained  from  the  questionnaires  is  intended  for  use  by  federal 
officials  and  PSRO  leadership  in  highlighting  design,  planning  and  opera- 
tion activities  that  must  be  accomplished  in  establishing  the  foundation 
for  a  successful  ambulatory  review  program. 

The  First  Year  Questionnaire:  Planning 

The  questionnaire  covered  four  major  planning  areas:    1)  development 
of  the  basic  conceptual  design  of  the  review  system,  2)  project  manage- 
ment (e.g.,  formation  and  responsibilities  of  the  project  management 
committee  and  project  staffing  requirements),  3)  demonstration  project 
relationships  with  other  PSRO  organizational  units  (the  PSRO  Board  and 
standing  committees),  and  4)  relationships  with  groups  and  organizations 
external  to  the  PSRO  (PSRO  members  and  providers  associated  with  the 
demonstration  project  as  volunteer  participants,  consultants,  and  the 
analysis  contractor).    The  methods  used  to  construct,  administer,  and 
analyze  the  questionnaire  are  reported  on  pages  8-12  of  Appendix  1, 
"Interim  Report  Cooperative  Demonstration  and  Analysis  of  Ambulatory 
Care  Quality  Assurance  Review  Methodology  in  PSRO  Settings."    Data  was 
analyzed  across  all  PSRO  projects  and  findings  reported  below  reflect 
the  opinion  of  aggregated  demonstrated  project  participants. 
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The  Second  Year  Questionnaire:    Design  and  Operations 

The  second  questionnaire  focused  more  specifically  on  the  actual 
success  of  each  site's  project  design  and  resultant  operation.  A 
substantial  number  of  items  were  aimed  at  management  and  administra- 
tion issues.    Other  items  questioned  resources  and  funding  and  methods 
for  topic  selection,  criteria  development,  data  collection  and  pro- 
cessing, review  and  decision  making,  and  intervention.    The  questionnaire 
was  divided  into  two  sections.    Whereas  the  first  year  questionnaire 
was  analyzed  across  all  PSRO  projects,  the  second  year  questionnaire 
was  analyzed  both  by  individual  project  (Section  I)  and  across  projects 
(Section  II).    The  findings  from  Section  I  are  reported  below;  the 
findings  from  Section  II  are  found  in  Volume  II  of  this  report. 

SYNOPSIS  OF  FIRST  YEAR  FINDINGS 

Twenty  requisites  and  five  additional  elements  for  a  planning 

process  were  extracted  from  interviews  with  25  ambulatory  care  quality 

review  system  planners  participating  in  five  demonstration  projects. 

Overall,  20  requisites  (numbered)  for  planning  were  identified  and 

five  elements  (E)  came  very  close  to  meeting  requisite  criteria. 

1.     Involve  the  PSRO  Board  in  the  review  of  progress  reports 
concerning  ambulatory  care  quality  assurance  system 
activities. 

El.      Prior  approval  of  ambulatory  care  quality  assurance 
objectives  by  the  PSRO  Board  is  essential  to  avoid 
committing  resources  to  activities  that  are  later 
abandoned. 

Two  respondents  said  that  PSRO  Board  approval  for  their  demonstra- 
tion projects  was  obtained  on  the  basis  of  little  information  on  Board 
members'  part.    Since  original  approval  of  the  project,  according  to 
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ten  respondents,  the  Board  has  not  been  active  in  project  activities 
other  than  receiving  reports,  primarily  at  Board  meetings.  Health 
Standards  and  Quality  Bureau  (HSQB)  required  Board  approval  of  PSRO 
participation  in  the  demonstration  program.    The  one  PSRO  which  did  not 
continue  to  participate  in  the  program  did  not  obtain  prior  Board 
approval.    Two  respondents  from  this  project  emphasized  the  need  for 
Board  agreement  and  approval  of  project  objectives  before  the  initia- 
tion of  activities  in  order  to  avoid  wasting  resources. 

2.  Assure  that  all  participants  in  review  system  design  and 
operation  (PSRO  staff,  PSRO  consultants,  the  funding 
agency  and  the  analysis  contractor)  have  a  complete  and 
common  understanding  of  PSRO  ambulatory  care  quality 
assurance  system  goals,  objectives  and  philosophy. 

Eight  respondents  emphasized  the  need  for  all  review  system 
planners  (PSRO  professionals,  staff  and  consultants  to  the  PSRO)  to 
have  an  early  understanding  of  the  goals.    They  felt  this  might  keep 
activities  peripheral  to  the  goals  from  draining  resources  at  a  later 
date.    One  respondent  emphasized  the  need  to  put  the  goals  in  written 
form  to  serve  as  an  unambiguous  reminder  to  all  throughout  the  review 
system  planning  phase.    Another  thought  that  a  common  understanding  was 
particularly  important  when  the  personnel  from  more  than  one  organization 
are  involved  in  project  design  and  planning.    Respondents  also  emphasized 
the  need  for  a  common  understanding  between  project  participants,  the 
funding  agency  and  the  analysis  contractor. 

3.  Attract  a  core  individual  or  individuals  to  the  review 
system  to  assure  the  continuing  momentum  of  review 
system  design. 

Five  respondents  mentioned  the  necessity  of  involving  a  person 
or  persons  with  both  a  well -formulated  review  system  core  idea  and 
the  time  and  dedication  to  devote  to  nursing  that  concept  through 
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the  stages  of  review  system  planning.    Such  a  dedicated  leadership 
is  required  to  provide  continuity  and  assure  continuing  momentum, 
and  particularly  to  refine  the  conceptual  base  of  the  review  system 
as  project  planning  progresses.    These  critical  persons  are  required 
for  assembling  staff  resources  and  for  interaction  with  the  various 
physician  groups  during  the  review  system  planning  phase.    One  of  the 
respondents  said  that  as  planning  progresses  and  project  staff  other 
than  core  individuals  begin  to  understand  the  details  of  the  review 
system  concept,  the  dependence  of  project  planning  on  this  core  declines. 

4.  Attract  to  the  review  system  planning  team  a  medical 
leader  whose  opinion  is  valued  by  community  physicians. 

Four  respondents  emphasized  that  the  involvement  of  a  medical 
leader  whose  opinion  is  respected  by  community  physicians  and  by  all 
project  planners  was  important  to  the  planning  phase.    The  medical 
leader's  importance  lies  in  his  ability  to  coordinate  the  sometimes 
diverging  perspectives  of  the  various  project  planners.    One  person 
who  speaks  with  authority  is  required  to  resolve  potential  conflicts 
and  demonstrate  to  planners  the  wisdom  of  certain  resolutions. 

5.  Define  consultant  role  and  tasks  as  specifically  as 
possible  to  maximize  consultant  usefulness  to  review 
system  planning. 

Six  respondents  felt  that  consultants  outside  the  PSRO  could  offer 
significant  assistance  to  ambulatory  care  review  system  planners.  Two 
respondents  emphasized  the  need  for  the  PSRO  to  specifically  define  the 
consultant's  tasks  and  expectations  early  in  the  PSRO-consultant  relationsh 

Three  review  system  planners  provided  suggestions  on  how  to  enhance 
the  usefulness  of  the  analysis  contractor  in  his  role  as  technical  con- 
sultant during  the  planning  process.    While  these  respondents  felt  that 
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the  analysis  contractor  was  quite  helpful  in  identifying  areas  of  the 
project  that  required  clarification,  they  thought  that  the  contractor 
should  have  taken  a  more  active  role  in  helping  the  planners  to  resolve 
these  issues.    While  these  particular  respondents  did  not  specifically 
so  mention,  two  others  mentioned  the  need  to  clarify,  at  an  early  date, 
the  role  to  be  assumed  by  the  analysis  contractor,  especially  in  the 
area  of  technical  consultation. 

6.  Designate  one  review  system  planning  team  member  to 
coordinate  and  rewrite  planning  documents,  integrating 
sections  produced  by  various  other  system  designers. 

7.  Include  on  the  review  system  planning  staff  one  person 
at  the  full-time  or  nearly  full-time  level  responsible 
for  coordinating  the  technical  activities  for  daily 
review  system  planning. 

E2       It  is  useful  to  include  on  the  review  system  planning 

team  a  person  with  skills  in  system  design  and  knowledge- 
able in  the  daily  operation  of  review  system  activities. 

Six  respondents  saw  the  need  to  include  a  full-time  or  nearly  full- 
time  PSRO  staff  member  to  coordinate  planning  and  manage  eht  operation 
of  the  review  system;  also,  as  one  respondent  mentioned,  this  full- 
time  staff  member  is  needed  to  attend  to  the  time-consuming  tasks  and 
the  myriad  of  details  that  arise  in  such  an  enterprise.    Three  respondents 
emphasized  the  importance  of  this  person  having  skills  in  system  design 
and  management  of  day-to-day  activities. 

8.  Emphasize  to  review  system  planners  that  planning  activities 
are  evolutionary  and  that  they  can  expect  testing  and 
revision  to  continue  beyond  the  initial  drafting  of  a  system 
design. 

Seven  planners  mentioned  the  difficulty  of  producing  planning 
documents  that  anticipated  the  future  details  of  their  proposed  ambu- 
latory care  review  systems.    Five  respondents  mentioned  that  planning 
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activities  would  continue  beyond  the  planning  phase  and  into  the  opera- 
tional phase.    They  emphasized  that  it  is  impossible  during  the  plan- 
ning phase  to  anticipate  all  the  contingencies  that  might  develop  in 
the  operation  of  the  review  system.    One  respondent  said  that  concept 
development  is  evolutionary,  requiring  modification  as  difficulties 
arise.    Others  said  that  alternative  solutions  to  problems  that  are 
anticipated  during  planning  can  often  only  be  investigated  in  an  opera- 
tional environment.    Two  respondents  mentioned  that  while  the  general 
review  system  design  and  initial  steps  can  be  finalized  during  plan- 
ning, the  details  must  await  operational  activities.    One  respondent 
thought  review  system  design  might  entail  a  trial-and-error  process 
for  certain  problems. 

9.      Implement  selected  feasibility  tests  of  the  review 
system  components  to  enhance  planning  prior  to 
committing  resources  to  larger-scale  operational 
activities. 

Nine  respondents  said  that  a  feasibility  test  of  some  of  the 
uncertainties,  utilizing  fewer  records  than  the  operational  review 
system  would  later  involve,  would  have  been  helpful.    Commonly  suggested 
review  system  components  for  feasibility  tests  were:    1)  the  abstract 
form,  and  2)  a  determination  of  what  data  were  likely  to  be  found  in 
the  ambulatory  record  and  might  thus  be  included  for  quality  review. 
Four  of  the  nine  respondents  thought  that  a  limited  pilot  study  of 
the  entire  review  system  would  have  been  helpful,  although  several 
viewed  the  demonstration  project  itself  as  a  pilot  study.    The  other 
five  respondents  felt  that  since  a  limited  amount  of  time  was  provided 
for  the  demonstration  project,  a  pilot  study  of  the  entire  review 
system  was  not  possible;  that  the  preparation  required  for  such  a 
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test  was  as  extensive  as  was  required  for  operating  the  actual  demon- 
stration review  system  and  was,  therefore,  not  an  efficient  use  of 
project  resources. 

10.     Make  review  system  staff  aware  of  the  purpose  and  demands 
of  planning  in  order  that  these  demands  do  not  dampen 
their  enthusiasm. 

E3.     Specific  deadlines  for  review  system  planning  and  planning 
documents  tend  to  expedite  those  activities  within  busy 
PSROs. 

Two  respondents  commented  on  a  related  issue,  the  effect  that 
extended  planning  periods  had  on  project  staff.    These  respondents  felt 
that  staff  enthusiasm  for  the  demonstration  project  was  jeopardized 
at  times  by  an  inability  to  investigate,  in  an  operational  setting, 
what  everyone  perceived  were  uncertainties  concerning  project  design. 
One  of  these  respondents  as  well  as  a  third  thought  that  project  inter- 
action with  outside  monitoring  (HSQB  and  the  analysis  contractor) 
should  be  more  intensive  over  a  shorter  period  of  time  in  order  to 
produce  an  improved  planning  process  and  allow  earlier  testing  of  the 
review  system  design. 

Responses  varied  concerning  the  amount  of  time  allotted  to  writing 
planning  documents.    Four  respondents  felt  that  a  sufficient  amount  of 
time  was  available  for  the  production  of  the  concept  paper,  the  pre- 
liminary and  the  final  plans.    Six  felt  that  they  could  have  used 
more  time.    Three  respondents  mentioned  the  difficult  and  time-consuming 
tasks  of  integrating  the  concerns  of  everyone  participating  in  project 
planning  (staff,  consultants,  HSQB  and  the  analysis  contractor)  into 
one  coherent  planning  document.    Three  of  those  who  felt  that  sufficient 
time  was  available  for  writing  documents  (planners  who  were  not  directly 
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involved  in  the  writing)  thought  that  if  more  time  had  been  allotted, 
the  document  writing  would  have  consumed  that  time  without  producing 
plans  of  a  higher  quality  or  containing  any  more  detail;  these  respon- 
dents felt  that  deadlines  served  the  function  of  expediting  organizational 
activities. 

11.      Include  on  a  review  system  management  committee  representa- 
tives of  those  physicians  whose  ambulatory  care  will  be 
reviewed. 

E4      Inclusion  of  key  physicians  skeptical  of  ambulatory  care 
quality  assurance  in  management  activities  can  help 
assure  that  their  skepticism  does  not  become  opposition. 

Each  of  the  five  PSROs  participating  in  the  demonstration  project 
formed  a  management  committee  to  oversee  the  planning  and  operation 
of  their  review  systems.    Five  respondents  mentioned  the  importance  of 
including  on  that  committee  representatives  of  the  physicians  whose 
medical  care  would  be  reviewed  so  that  a  liaison  could  be  established 
between  the  review  system  and  the  individual  physicians.    The  liaison, 
a  physician,  would  convey  the  concerns  of  his  constituents  to  the 
PSRO,  guaranteeing  communication  between  the  physician  community  and 
the  ambulatory  review  system  managers.    Representatives  from  hospital 
staffs,  according  to  the  five  respondents,  can  serve  as  the  liaison 
between  the  PSRO  and  community  physicians  since  most  physicians  are 
associated  with  a  hospital.    Two  respondents  mentioned  that  their 
review  system  management  committee  included  a  physician  who  had  a 
skeptical  attitude  toward  the  review  of  ambulatory  care  and  who  was 
recruited  for  the  committee  to  assure  that  his  skepticism  did  not 
become  opposition. 
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12.      Indicate  to  planners  that  they  should  expect  that  PSRO 
members  will  express  little  desire  to  become  involved 
in  review  system  planning  activities. 

Seven  review  system  planners  said  that  although  the  PSRO  general 
membership  was  notified,  it  expressed  little  desire  to  participate  in 
planning  activities.    Several  respondents  were  surprised  that  the 
physician  community  did  not  express  more  negative  reactions  concerning 
ambulatory  review.    Only  one  respondent  stated  that  a  "couple"  of  PSRO 
physicians  responded  by  volunteering  for  project  committees.  Another 
respondent  elicited  little  reaction  from  physicians  even  after  his 
personal  explanations  of  the  project.    Five  respondents  explained  that 
notification  usually  occurred  through  the  PSRO  newsletter,  although 
in  two  projects  the  PSRO  annual  meeting  was  also  used  to  inform  the 
membership.    In  either  case  only  a  general  project  description  was 
provided. 

E5       Inclusions  of  the  PSRO  membership  in  the  design  and 
operation  of  the  ambulatory  care  quality  assurance 
system  should  await  its  testing  in  order  not  to  arouse 
premature  opposition. 

Nine  respondents  felt  that  it  was  premature  at  this  time  to  involve 
the  PSRO  general  membership  in  the  details  of  the  design  and  operation 
of  the  ambulatory  care  review  system.    These  planners  felt  that  the 
membership  should  not  be  involved  until  review  was  perfected  and  a  well- 
developed  and  tested  system  could  be  presented  to  them.    Two  of  these 
planners  specifically  mentioned  the  danger  of  arousing  opposition  to 
ambulatory  review,  opposition  that  would  "muddy  the  waters"  for  the 
future  and  defeat  ambulatory  review  before  it  had  a  chance  to  develop. 

Sentiment  on  this  point  was  not  unanimous.    One  respondent  felt 
that  the  benefits  to  be  gained  by  increased  membership  interest  in 
the  PSRO  and  the  ambulatory  project  were  worth  risking  the  potential 
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opposition  that  might  occur  through  such  involvement.    Another  felt 
that  should  disharmony  develop,  project  planning  should  include  the 
"loyal  opposition."    Another  respondent  from  the  same  project  felt 
that  as  many  physician  groups  as  possible  should  participate  in  order 
to  legitimize  the  project  within  the  medical  community. 

13.  Recruit  from  the  review  system  and  employ  as  representa- 
tives to  the  physician  community  those  physicians  with 
strong  personal  ties  to  the  groups  of  physicians  whose 
ambulatory  care  will  be  reviewed. 

14.  Identify  local  medical  community  opinion  molders  and 
enlist  them  as  personal  ambassadors  from  the  PSRO. 

Thirteen  respondents  emphasized  that  a  critical  factor  in  gain- 
ing physician  participation  in  the  demonstration  project  was  a  personal 
relationship  between  a  physician  representing  the  review  system  and 
the  physicians  whose  ambulatory  care  was  to  be  reviewed.    In  all  the 
projects  this  personal  relationship  was  one  that  had  developed  prior 
to  the  demonstration  project.    It  was  employed  in  obtaining  physician 
volunteers  for  the  project  through  letters,  telephone  calls  and  actual 
personal  contracts.    In  the  two  projects  associated  with  hospitals 
representatives  of  medical  staffs  were  used  to  encourage  physician 
participation  in  the  demonstration  project.    Members  of  committees  in 
one  project  were  selected  in  large  measure  on  the  basis  of  their 
potential  ability  to  gain  the  cooperation  of  physician  volunteers. 
One  respondent  said  that  had  physicians  been  asked  anonymously  to 
volunteer  to  have  their  ambulatory  care  reviewed  they  would  have 
refused.    The  "peer  pressure"  implied  by  personal  contact  with  a  PSRO 
physician  representative  generated  the  cooperation.    Two  other  respon- 
dents mentioned  that  securing  the  cooperation  of  physicians  in  their 
areas  required  repeated  personal  contact  with  the  potential  volunteers. 
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One  project,  in  an  effort  to  obtain  the  participation  of  one  physician 
in  a  small  group  practice,  actually  obtained  the  participation  of  all 
members  of  that  group  through  contact  with  the  one  group  member.  Five 
respondents  saw  the  need  for  PSROs  to  identify  the  opinion  molders  in 
each  locality  who  would  serve  as  personal  ambassadors  from  the  PSRO  to 
the  physician  and  generate  support  for  ambulatory  care  quality  assurance 
activities.    Permission  to  review  the  ambulatory  care  of  physicians 
practicing  in  the  outpatient  department  of  a  project  hospital  was  secured 
through  the  hospital  administration,  and  each  individual  physician  did 
not  need  to  be  recruited. 

15.  Emphasize  to  the  participating  physician  community  that 
the  review  system  will  provide  the  physician  with  informa- 
tion useful  to  his  practice. 

Five  respondents  felt  that  one  of  the  best  methods,  of  eliciting 
cooperation  from  volunteer  physicians  whose  care  will  be  reviewed  is 
to  agree  to  provide  the  physicians  with  feedback  of  review  results. 
One  respondent  said  that  volunteers  feel  that  review  may  be  used  to 
gain  some  knowledge  about  their  practices  without  any  expense  on  their 
part.    Another  felt  that  it  was  important  to  provide  feedback  as  quickly 
as  possible.    The  importance  of  providing  positive  as  well  as  negative 
feedback  to  the  physician  was  also  emphasized.    Some  physician  volunteers, 
according  to  one  respondent,  made  the  provision  of  feedback  a  condition 
for  their  participation. 

16.  Emphasize  to  those  physicians  whose  ambulatory  care  will 
be  reviewed  by  the  quality  assurance  system  that  their 
early  and  willing  cooperation  will  avoid  the  imposition 
of  a  system  from  outside  the  local  medical  community. 
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Fourteen  respondents  felt  that  an  important  factor  in  attracting 
volunteers  to  ambulatory  review  activities  was  the  desire  of  the  physi- 
cians themselves  to  become  involved  in  a  physician-led  effort  to  estab- 
lish ambulatory  care  quality  assurance  systems,  thus  avoiding  nonphysician 
design  of  systems  that  may  possibly  have  antithetical  consequences  for 
them.    Two  respondents  specifically  mentioned  physicians'  concern  with 
the  imposition  of  an  unworkable  system  by  the  federal  government,  one 
not  attumed  to  local  needs.    Two  other  respondents  said  that  the  PSRO 
emphasized  to  potential  volunteers  the  opportunity  to  influence,  through 
the  demonstration  project,  the  evolving  federal  policy  toward  ambulatory 
care  quality  assurance.    One  of  these  respondents  informed  volunteers 
of  previous  PSRO  experiences  in  influencing  policy. 

17.      Prepare  system  planners  to  expect  that  physicians  whose 
care  is  being  reviewed  require  few  details  concerning 
the  technical  operation  of  the  review  system  during  the 
planning  phase. 

Seven  respondents  mentioned  that  the  physician  volunteers  requested 
general,  not  detailed,  information  concerning  the  proposed  review  system. 
This  information  was  supplied  to  them  in  both  written  and  verbal  form. 
Two  respondents  from  different  projects  mentioned  that  a  personal  con- 
tact session  of  only  10  or  15  minutes  was  necessary  to  explain  project 
objectives  and  procedures,  and  few  volunteers  requested  additional 
detail.    One  respondent  mentioned  that  volunteers  were  more  concerned 
with  the  degree  to  which  review  system  activities  would  affect  them 
than  with  the  system's  technical  details.    Another  respondent  said 
that  volunteers  were  concerned  about  what  the  review  system  would  cost 
them.    Volunteers  were  interested  in  what  criteria  would  be  used  in 
evaluating  their  ambulatory  care.    A  respondent  from  a  project  associated 
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with  PSRO  area  hospitals  confirmed  that  the  hospitals  were  also  inter- 
ested in  a  general,  and  not  a  detailed,  description  of  the  proposed 
review  system. 

18.  Prepare  system  planners  to  expect  that  physician  volun- 
teers whose  ambulatory  care  is  being  reviewed  will  express 
little  interest  in  participating  in  the  design  or  the 
technical  activities  of  operating  the  review  system. 

Six  respondents  said  that  the  physician  volunteers  were  not  inter- 
ested in  participating  in  the  design  or  operation  of  the  ambulatory 
care  review  system.    Five  of  these  said  that  none  of  the  volunteers 
expressed  any  desire  to  become  involved.    The  sixth  respondent  said 
that  a  few  did  express  a  desire,  while  other  physician  volunteers 
offered  suggestions.    The  majority  of  volunteers,  however,  were  not 
interested  in  the  technical  details  of  the  review  system.    One  respon- 
dent felt  that  the  physicians  were  comfortable  in  delegating  this 
responsibility  to  their  project  representative.    Another  felt  that  they 
simply  did  not  have  the  time  for  such  activities  and  that  some  mechanism 
should  be  created  to  allow  them  the  level  of  involvement  that  their 
time  permits. 

19.  Assure  that  review  system  activities  do  not  interfere  with 
the  physician's  office  routine. 

Five  respondents  mentioned  that  physician  volunteers  concerned 
that  review  system  activities  might  interfere  with  office  routine. 
One  potential  volunteer  decided  not  to  participate  in  the  demonstra- 
tion project  because  of  what  he  saw  as  potential  disruption  to  his 
practice.    Two  respondents  said  that  this  project  goal  was  to  demon- 
strate that  effective  quality  assurance  systems  can  be  developed  and 
operated  without  interfering  with  office  routine. 
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20.     Assure  the  confidentiality  of  all  physician  and  patient 
information. 

Nine  respondents  mentioned  that  physician  volunteers  were  con- 
cerned that  the  medical  record  data  remain  confidential.    They  were 
apprehensive  about  the  potential  dangers  involved  in  breaching  patient 
privacy  by  releasing  information.    Another  concern  was  whether  the 
physician  maintained  legal  custody  of  the  medical  information  and 
could  release  it  for  quality  review  activities  without  risking  any 
patient-initiated  legal  consequences.    One  respondent  said  that  volun- 
teers were  most  concerned  about  the  identification  of  the  physician. 
Another  mentioned  that  the  PSRO  anticipated  the  problem  and  had  assured 
confidentiality  of  all  data  in  the  initial  PSRO  letter  inviting  physi- 
cians to  participate  in  the  project.    Still  another  stated  that  this 
potential  problem  was  mitigated  by  PSRO  confidentiality  policies.  Two 
physician  respondents  felt  that  the  confidentiality  issue  was  not  the 
real  issue  but  was  being  used  by  some  as  an  excuse  for  not  participating 
in  quality  assurance  activities. 
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SECOND  YEAR  FINDINGS 
Method 

Subjects  and  Questionnaire  Development 

The  subjects  for  the  survey  were  87  persons  who  were  involved  in 
various  aspects  of  the  demonstration  projects  in  the  five  sites.  For 
analysis  purposes,  subjects  were  classified  according  to  their  role  in 
the  demonstration  project  (e.g.,  PSRO  staff  or  physician,  physician  or 
nonphysician  consultant,  abstractor  and  other).    Also,  they  were  classified 
according  to  their  level  and  type  of  involvement  in  the  project. 

The  questionnaire  was  composed  of  two  sections.    The  first  section 
had  18  objective  items  where  the  respondent  was  requested  to  circle  one 
of  three  answer  choices  which  best  described  their  position  and  then 
indicate  the  strength  of  their  position  along  the  continuum  from  strongly 
agree  to  strongly  disagree.    If  respondents  felt  strongly  about  a  question, 
they  had  the  option  of  answering  more  than  one  choice. 

The  second  section  is  comprised  of  a  mixture  of  22  open  and  closed 
questions.    The  bulk  of  these  questions  focus  on  describing  the  PSRO 
demonstration  experience  to  provide  technical  assistance  to  new  PSROs 
beginning  ambulatory  care  review  programs. 

The  questionnaire  development  began  with  staff  review  of  over  150 
articles,  books,  and  papers  on  the  topic  of  quality  assurance.  Each 
reference  was  read  to  extract  relevant  information  on  ambulatory  care 
quality  assurance.    A  listing  of  strengths  and  weaknesses  in  conducting 
ambulatory  quality  assurance  was  comprised.    Special  problems  in  the 
evaluation  of  ambulatory  care  were  also  noted  within  the  literature. 
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HCMS'  previous  experience  with  ambulatory  care  quality  assurance 
projects  was  then  used  to  build  a  model  for  reviewing  care  rendered  in 
the  office  setting.    The  model  was  comprised  of  three  cycles:  screening, 
action,  and  restudy.    The  components  of  the  model  consisted  of  such 
factors  as  selection  of  topics  or  providers,  criteria  development, 
review  and  decision  making,  and  impact  assessment. 

From  this  model  HCMS  staff  developed  a  list  of  issues  being  investi- 
gated within  each  site.    These  issue  lists  were  site  specific  but  gave  a 
picture  of  the  issues  and  problem  areas  facing  each  of  the  demonstration 
sites.    From  the  issue  lists  a  common  thread  of  issues  across  all  sites 
was  determined.    HCMS  and  HSQB  staff  discussions  were  also  held  to 
review  the  entire  ambulatory  care  demonstration  evaluation  effort.  Such 
discussions  when  combined  with  the  issue  lists  and  model  tended  to  focus 
survey  attention  on  the  future  of  ambulatory  care  rather  than  objectively 
determining  what  each  site  did  or  didn't  do  and  why. 

From  this  process  a  list  of  76  questions  were  developed.  These 
questions  were  all  open  ended.    In  order  to  better  quantify  some  of  them 
and  eliminate  others,  a  sample  of  32  questions  was  discussed  face  to 
face  with  a  group  of  people  working  on  the  demonstration  at  one  of  the 
sites.    HCMS  staff  reviewed  the  interpretations  given  to  the  questions 
as  well  as  the  responses.    The  interpretations  and  points  of  uncertainty 
displayed  by  PSRO  personnel  were  then  utilized  by  HCMS  staff  to  revise 
the  list  of  questions  and  convert  others  to  objective  (closed  ended) 
questions.    This  process  also  assisted  in  determining  relevance  of 
questions  as  well. 
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A  draft  of  the  survey  was  prepared  in  late  August,  1978.    The  HCMS 
staff  critically  reviewed  the  questions  and  a  final  draft  was  prepared 
on  September  1,  1978.    A  letter  soliciting  cooperation  support  from  Dr. 
Michael  Goran  of  HSQB  and  a  personal  letter  with  directions  for  survey 
dissemination  were  mailed  to  each  of  the  Executive  Directors  within  each 
site.    Each  Executive  Director  was  then  requested  to  disseminate  the 
survey  together  with  a  self-addressed  stamped  folder  to  each  of  the  key 
people  involved  in  the  project.    The  list  of  people  were  derived  after 
HCMS  reviewed  cost  log  voucher  data  from  each  site  and  confirmed  each 
list  of  names  with  each  Executive  Director. 

Complete  packages  of  surveys  and  return  envelopes  were  mailed  to 
each  Executive  Director  in  the  second  week  of  September.    A  two-week 
turn  around  was  requested  in  the  letter  which  accompanied  each  of  the 
surveys. 

A  series  of  follow-up  calls  from  HCMS  were  required  to  obtain 
responses  from  each  of  the  sites.    After  seven  weeks,  data  analysis 
began  and  no  more  questionnaires  were  accepted. 

The  final  response  rate  was  44  or  51%.    The  majority  of  those 
nonrespondents  were  physicians  who  served  as  consultants  to  the  project 
in  its  beginning  stages. 

Data  Analysis 

For  each  individual  project,  responses  in  Section  I  of  the  question- 
naire were  tallied  on  a  three-point  scale;  choices  1  and  2  (agree)  and 
4  and  5  (disagree)  were  collapsed.    Since  several  items  were  mutually 
exclusive,  if  a  respondent  had  conflicting  responses  indicating  he  did 
not  understand  the  item,  these  responses  were  eliminated  from  the  data 
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base.    From  a  total  response  rate  of  1,274  the  usable  responses  totaled 
1,203.    For  each  item,  the  response  rate  for  agree,  undecided,  and 
disagree  is  reported. 

Additional  analysis  examined  differences  between  physician  and 
nonphysician  responses  for  those  items  where  there  was  at  least  two 
responses  in  both  the  agree  and  disagree  categories.    The  analysis 
included  only  those  individuals  who  were  classified  as  either  a  PSRO 
staff  physician  or  physician  consultant  and  a  PSRO  staff  nonphysician, 
nonphysician  consultant  or  subcontractor  consultant.    Also  included 
were  only  those  individuals  who  were  rated  as  being  involved  in  all 
or  at  least  four  phases  of  demonstration  development. 

Analysis  of  Section  II  of  the  questionnaire  included  reading  the 
open-ended  responses  and  synthesizing  meaning  into  appropriate  cate- 
gories, depending  on  the  nature  of  the  question.    The  close-ended 
questions  in  Section  II  were  tallied  with  no  changes  in  the  response 
categories. 
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Discussion  of  Survey  Findings  (Section  I) 

Presented  in  Table  2,  "Survey  Items  and  Responses:  Multnomah 
Foundation  for  Medical  Care"  are  the  18  questions  along  with  the  total 
responses  per  question.    Thirteen  questionnaires  were  sent  to  the  PSRO 
and  eight  were  returned.    The  total  responses  were  263,  14  of  which  were 
not  useable,  leaving  total  useable  responses  of  249. 

Respondents  agreed  that  their  project  was  aimed  primarily  at 
research  while  three  respondents  believed  that  the  project  was  a  com- 
bination of  both  research  and  implementation  of  an  operational  review 
system  (Item  1  and  2).    Respondents  were  split  of  whether  to  rely  on 
others  research  or  create  their  own  concepts  which  accommodate  the 
unique  characteristics  of  their  geographic  area  (Item  5).    The  majority 
of  respondents  believe  that  use  of  voluntary  physician  participants  is 
necessary  even  though  it  may  bias  research  findings  (Item  3). 

In  regards  to  the  goals  of  the  project,  it  was  agreed  that  the 
purpose  of  a  PSRO  ambulatory  care  review  system  is  to  improve  care 
rendered  to  all  patients  as  well  as  to  help  contain  health  care  costs; 
cost  effectiveness  must  be  emphasized  if  the  PSRO  program  is  to  sur- 
vive (Items  4  and  11).    Respondents  are  unable  to  say  whether  it  is 
possible  to  make  cost-effective  programs,  however,  they  believe  that 
both  deviant  physician  practice  patterns  and  the  entire  mainstream 
of  care  should  be  reviewed  (Item  9).    Topic  selection  should  focus 
on  those  topics  which  have  the  greatest  potential  for  improving  the 
patients  health,  and  depend  on  data  accessibility.    Respondents  did 
not  agree  on  whether  topics  are  best  developed  by  physician  members 
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or  PSRO  peer  review  committees  (Item  12).    Personnel  agreed  that  review 
of  nonphysician  services  will  become  necessary  in  the  future,  however, 
are  not  sure  whether  that  is  feasible  immediately  (Item  14).  Respond- 
ents are  split  on  whether  criteria  should  focus  on  over-  and  under- 
utilization,  but  agree  unamiously  that  the  ultimate  goal  of  a  review 
system  is  to  develop  standards  and  criteria,  compare  practice  patterns 
to  them  and  then  encourage  compliance  where  variations  exist  (Items  17 
and  18).    Respondents  were  split  on  whether  technical  assistance  from 
outside  the  PSRO  was  needed  (Item  6). 

The  feasibility  issue  was  raised  in  several  questions.    There  is 
split  agreement  as  to  whether  ambulatory  care  review  is  easily  feasible, 
both  in  general  and  on  a  nationwide  basis  (Items  8,  13  and  15).    It  was 
unanamiously  agreed  that  further  research  is  needed  on  the  best  approach 
to  changing  physician  practice  patterns  and  there  was  split  agreement 
as  to  whether  direct  intervention  with  individuals  or  intervention 
with  groups  of  physicians  with  no  individual  comparisons  is  most  effec- 
tive (Item  10). 

An  additional  analysis  was  conducted  on  those  questions  which  showed 
disagreement  among  respondents.    The  items  were  analyzed  where  two  or 
more  respondents  appeared  to  disagree  (e.g.,  at  least  two  or  more 
responses  in  the  "agree"  category  and  at  least  two  or  more  responses 
in  the  "disagree"  category).    Analysis  was  conducted  to  determine 
whether  disagreement  may  have  occurred  due  to  difference  of  opinion 
between  physicians  and  nonphysicians  who  had  been  heavily  involved  in 
the  entire  project.    Respondents  substantially  disagreed  on  ten  items 
(56%),  however  analysis  did  not  clearly  indicate  differences  between 
the  physicians  and  nonphysicians. 

41 


TABLE  2.    SURVEY  ITEMS  AND  RESPONSES: 


MULTNOMAH  FOUNDATION  FOR  MEDICAL  CARE 


Our  demonstration  is  a  research  project  wherein  data  collection 
will  assist  in  performing  subsequent  ambulatory  care  studies. 

 TOTAL  RESPONSES  

Agree      Undecided  Disagree 

7  0  0 

We  are  interested  only  in  the  development  of  an  operational 
ambulatory  care  review  system  for  improving  quality  of  care 
rendered  in  our  area. 

 TOTAL  RESPONSES  

Agree      Undecided  Disagree 

0  1  2 

Our  demonstration  is  a  combination  of  research  design  and 
implementation  of  an  operational  review  system.    We  believe 
one  must  conduct  adequate  front  end  investigation  prior  to 
implementation. 

 TOTAL  RESPONSES  

Agree      Undecided  Disagree 

3  0  0 

In  view  of  the  problems  in  measurement,  conducting  research  on 
ambulatory  care  is  unlikely  to  produce  meaningful  results. 

And 

The  prime  emphasis  of  any  PSRO  should  be  on  development  of  an 
operational  quality  assurance  project,  research  is  not  likely 
to  result  in  improved  patient  care  in  the  practice  settings. 

 TOTAL  RESPONSES  

Agree       Undecided  Disagree 

2  2  3 

A  PSRO  must  conduct  research  to  better  understand  the  problem 
before  an  operational  program  to  resolve  the  program  can  be 
implemented. 

 TOTAL  RESPONSES  

Agree      Undecided  Disagree 

6  0  0 
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a)  In  our  ambulatory  demonstration  project,  the  use  of  volunteer 
subjects  has  been  invaluable.  No  study  of  ambulatory  quality 
assurance  could  be  completed  without  voluntary  participants. 

TOTAL  RESPONSES  

Agree      Undecided  Disagree 

4  1  1 

b)  Using  physicians  who  volunteer  their  office  data  is  a  practical 
solution  to  a  practical  problem  (obtaining  practice  data  from 

a  physician's  office).    In  most  instances,  there  is  no  other 
choice. 

 TOTAL  RESPONSES  

Agree      Undecided  Disagree 

4  0  0 

c)  The  use  of  volunteers  introduces  a  bias  into  the  results  of 
any  study  which  limit  the  general izability  of  the  findings  to 
the  practices  studied  and  reduces  the  potential  for  using 
results  in  HSQB  policy  development. 

 TOTAL  RESPONSES  

Agree      Undecided  Disagree 

3  1  1  ' 

The  purpose  of  a  well  constructed  PSRO  ambulatory  care  review  system 

a)  To  improve  the  care  rendered  to  all  patients  in  its  area. 

 TOTAL  RESPONSES  

Agree      Undecided  Disagree 

4  0  1 

b)  Change  physician  practice  patterns  which,  in  turn,  would 
impact  upon  the  care  rendered  to  patients  in  the  area. 

 TOTAL  RESPONSES  

Agree      Undecided  Disagree 

2  1  1 

c)  To  improve  the  quality  of  care  rendered  to  patients  in  the 
office  setting  as  well  as  to  help  contain  health  care  costs. 

 TOTAL  RESPONSES  

Agree      Undecided  Disagree 

5  0  1 
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5.  a)     In  conducting  an  ambulatory  care  quality  assurance  demon- 

stration, reliance  on  the  research  findings  of  others  is  use- 
ful but  not  mandatory. 

And 

b)  PSROs  beginning  ambulatory  projects  should  place  little  reliance 
on  the  work  of  others  in  establishing  a  review  program,  most 

of  our  concepts  were  self  generated  to  accommodate  the  unique 
characteristics  of  our  geographic  area. 

 TOTAL  RESPONSES  

Agree      Undecided  Disagree 

5  2  6 

c)  It  is  necessary  to  conduct  a  complete  literature  search  to 
review  the  findings  of  others  prior  to  starting  any  research 
or  demonstration  project. 

 TOTAL  RESPONSES  

Agree      Undecided  Disagree 

2  0  2 

6.  a)     Our  staff  resources  were  such  that  little  or  no  technical 

assistance  from  outside  the  PSRO  was  needed. 

 TOTAL  RESPONSES  

Agree      Undecided  Disagree 

4  1  3 

b)  In  launching  an  ambulatory  care  demonstration  project  heavy 
emphasis  should  be  placed  upon  the  expertise  of  various  out- 
side consultants  for  the  first  two  years. 

 TOTAL  RESPONSES  

Agree       Undecided  Disagree 

1  1  1 

c)  The  technical  assistance  provided  through  HSQB  and  HCMS  served 
all  of  our  technical  assistance  needs  during  the  demonstration. 

 TOTAL  RESPONSES  

Agree       Undecided  Disagree 

0  2  2 
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The  amount  of  seriously  substandard  ambulatory  care  cannot 
accurately  be  assessed  based  upon  our  demonstration  experience 
thus  far.    It  is  too  soon  to  tell. 

TOTAL  RESPONSES  

Agree      Undecided  Disagree 

3  2  0 

Substandard  care  in  the  ambulatory  sector  is  pervasive  and  can 
be  found  to  a  greater  or  lesser  degree  in  most  office  practices. 

TOTAL  RESPONSES  

Agree      Undecided  Disagree 

0  1  2 

From  our  experience  thus  far  it  appears  that  the  degree  of 
substandard  ambulatory  care  is  fairly  low  reflecting  only  a 
small  percentage  of  physicians. 

 TOTAL  RESPONSES  

Agree      Undecided  Disagree 

6  0  0 

Many  illnesses  treated  in  the  ambulatory  sector  are  ill 
defined,  self-1 imiting  conditions  for  which  there  are  no 
clearly  accepted  therapeutic  regimes.    As  such,  review  of 
ambulatory  care  is  not  feasible. 

 TOTAL  RESPONSES  

Agree      Undecided  Disagree 

3  1  1 

Quality  assurance  in  ambulatory  care  is  no  different  than  any 
other  form  of  care,  it  exists  in  finite  amounts  and  those 
amounts  can  be  measured  just  the  same  as  inpatient  care  can 
be. 

 TOTAL  RESPONSES  

Agree       Undecided  Disagree 

2  1  1 

From  our  experience  as  an  ambulatory  demonstration  site,  it 
seems  quite  feasible  to  implement  quality  assurance  programs 
for  ambulatory  care  on  a  nationwide  basis. 

 TOTAL  RESPONSES  

Agree       Undecided  Disagree 

2  1  2 
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a)     An  ambulatory  quality  assurance  program  should  look  at  just 
those  physician  practice  patterns  which  differ  significantly 
from  practice  norms  for  each  area. 

TOTAL  RESPONSES 


Agree      Undecided  Disagree 
1  0  2 


b)     An  ambulatory  quality  assurance  program  should  look  at  the 
entire  mainstream  of  care  and  seek  to  optimize  the  practices 
of  as  many  doctors  as  possible. 

TOTAL  RESPONSES 


Agree      Undecided  Disagree 
2  0  0 


c)     At  this  time,  it  is  not  possible  to  say  whether  ambulatory 
quality  assurance  programs  can  be  made  cost  effective  enough 
to  review  the  entire  spectrum  of  ambulatory  care. 


TOTAL  RESPONSES 


Agree      Undecided  Disagree 
7  1  0 


10.     a)     Changing  practice  behavior  is  best  achieved  by  direct  practice 
pattern  feedback  to  each  practitioner  being  reviewed. 


TOTAL  RESPONSES 


Agree      Undecided  Disagree 
4  1  0 


b)     Feedback  comparing  individual  practices  to  a  norm  or  standard 
results  in  physician  defensi veness  and  resistance.    The  best 
way  to  change  practice  behavior  is  through  education  and 
general  feedback  to  groups  of  physicians  without  individual 
comparisons. 

TOTAL  RESPONSES 


Agree      Undecided  Disagree 
2  2  3 


c)     The  best  approach  to  changing  physician  practice  patterns  for 
purposes  of  quality  assurance  is  not  known  at  present,  more 
research  on  this  issue  must  be  undertaken. 

TOTAL  RESPONSES 


Agree      Undecided  Disagree 
5  0  0 
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11.     a)     Cost  containment  and  quality  assurance  are  totally  mutually 
exclusive  and  cannot  be  dealt  with  simultaneously  in  ambu- 
latory quality  assurance  efforts. 

TOTAL  RESPONSES 


Agree      Undecided  Disagree 
1  0  3 


b)     The  primary  purpose  of  a  PSRO  quality  assurance  project  is  to 
improve  patients  care  regardless  of  the  cost  implications. 

TOTAL  RESPONSES 


Agree      Undecided  Disagree 
0  1  3 


c)     Ambulatory  quality  assurance  while  emphasizing  improved 
patient  outcomes  must  also  consider  cost  effectiveness  in 
patient  care  if  the  PSRO  program  is  to  survive. 

TOTAL  RESPONSES 


Agree      Undecided  Disagree 
7  0  0 


12.    a)     Ambulatory  care  review  should  focus  on  those  topics  which  have 
the  greatest  potential  for  improving  the  patient's  health. 

TOTAL  RESPONSES 


Agree      Undecided  Disagree 
3  1  0 


b)     Review  topics  in  ambulatory  care  must  be  chosen  according  to 
data  accessibility,  and  frequency  of  occurrence  in  the  primary 
setting. 

TOTAL  RESPONSES 


Agree      Undecided  Disagree 
6  0  1 


c)     Review  topics  are  best  developed  by  focusing  on  specific 

practice  problems  determined  by  physician  members  of  PSRO  peer 
review  committees. 

TOTAL  RESPONSES 


Agree      Undecided  Disagree 
4  0  2 


47 


13.  a)  The  approach  we  have  used  to  conduct  ambulatory  quality 
assurance  will  be  acceptable  and  feasible  in  conducting 
review  for  all  physicians. 

TOTAL  RESPONSES 


Agree      Undecided  Disagree 
2  3  2 


b)     Our  demonstration  approach  is  quite  unique  and  unlikely  to  be 
duplicated  to  the  point  where  it  would  be  acceptable  to 
physicians  in  other  PSRO  areas. 

TOTAL  RESPONSES 


Agree      Undecided  Disagree 
2  1  2 


c)     The  approach  we  used  in  conducting  ambulatory  quality  assur- 
ance has  been  largely  unacceptable  to  physicians. 

TOTAL  RESPONSES 


Agree      Undecided  Disagree 
0  0  4 


14.    a)     The  integration  of  review  for  ambulatory  services  rendered  by 
nonphysicians  with  existing  PSRO  efforts  is  not  feasible  at 
this  time. 

TOTAL  RESPONSES 


Agree      Undecided  Disagree 
1  2  2 


b)     As  the  trend  toward  holistic  patient  care  increases,  the 
necessity  of  reviewing  nonphysician  treatment  will  become 
necessary  sometime  in  the  future. 

 TOTAL  RESPONSES 


Agree      Undecided  Disagree 
8  0  0 


c)     Incorporating  the  work  done  by  nonphysician  practitioners  into 
the  peer  review  process  is  quite  feasible  and  could  be  under- 
taken immediately. 

TOTAL  RESPONSES 


Agree      Undecided  Disagree 
2  1  1 
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15.    a)     The  procedures  for  ambulatory  quality  assurance  developed  in 
our  demonstration  are  transferrable  to  any  other  PSRO  without 
modification  or  special  arrangements. 

TOTAL  RESPONSES 


Agree      Undecided  Disagree 
2  0  3 


b)     The  nature  of  our  demonstration  is  quite  unique  and  it  is 

unlikely  that  our  approach  could  ever  be  duplicated  in  another 
PSRO. 

TOTAL  RESPONSES 


Agree      Undecided  Disagree 
0  1  2 


c)  Transferrence  of  our  ambulatory  quality  assurance  procedures 
to  other  PSROs  while  achievable  would  entail  much  effort  and 
very  probably  modification  of  arrangements  and  procedures. 

TOTAL  RESPONSES 


Agree      Undecided  Disagree 
5  1  0 


16.  Problems  uncovered  in  practices  with  respect  to  one  review 

topic  are  often  associated  with  problems  in  other  topics. 

TOTAL  RESPONSES 


Agree      Undecided  Disagree 
6  1  0 


17.  The  criteria  developed  for  each  review  topic  should  focus  upon 

over  and  underutilization  of  medical  services  simultaneously. 


TOTAL  RESPONSES 


Agree      Undecided  Disagree 
4  2  1 


18.  The  ultimate  goal  of  an  ambulatory  quality  assurance  review 

system  is  to  develop  standards  and  criteria,  compare  practice 
patterns  to  them  and  then  encourage  compliance  where  variations 
exist. 

TOTAL  RESPONSES 


Agree       Undecided  Disagree 
8  0  0 
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SECTION  IV 
QUANTITATIVE  PROGRAM  DESCRPTORS 
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METHOD 

Requests  for  quantitative  descriptions  tailored  to  fit  the  specific 
programs  of  each  of  the  five  projects  were  sent  out  in  August  1978. 
Requested  data  involved  population  and  activity  items  for  the  sites 
under  study  by  the  projects  including  such  items  as  the  number  of 
hospitals,  clinics,  physicians,  patients  and  patient  visits  at  these 
sites.    Tabulations  were  requested  of  physicians  by  specialty  and 
patients  by  source  of  payment.    In  addition  quantitative  descriptors  of 
the  review  activities  were  requested.    These  included  numbers  of  physi- 
cians and  patients  reviewed  by  topic  and  the  number  of  patient  visits 
involved.    Finally,  distribution  of  the  variations  or  exceptions  to 
review  criteria  were  requested  along  with  quantitative  description  of 
the  feedback  and  intervention  function  of  the  project.    Other  data  of 
specific  significance  to  individual  projects  was  also  included  in  the 
requests. 

Quantitative  data  reports  submitted  by  the  projects  in  response  to 
these  requests  were  screened  by  HCMS  personnel  for  completeness  and 
consistency  and  follow  up  was  made  by  telephone  to  make  the  data  as 
valid  as  possible.    Some  requested  data  could  not  be  supplied  by  the 
project  because  it  was  not  available  at  the  sites  studied.    Only  data 
available  as  of  August  31,  1978  were  used  and  some  sites  were  in  the 
midst  of  the  review  cycle  at  that  time.    The  quantitative  data  that  was 
submitted  by  the  five  projects  was  formatted  by  HCMS  staff  into  tabular 
form  designed  to  be  as  consistent  as  possible  among  projects  but  still 
displaying  the  unique  qualities  of  the  individual  project. 
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SITES  AND  PHYSICIANS  REVIEWED 

The  two  cycles  of  the  ambulatory  review  project  reported  here 
involved  a  total  of  34  physicians  practicing  at  18  sites  and  serving  an 
estimated  95,000  patients.    Fourteen  of  these  sites  were  solo  practices, 
the  largest  clinic,  comprising  ten  physicians,  served  an  estimated  30,000 
patients.    Physicians  at  this  site  were  reviewed  for  eight  topics, 
other  sites  ranged  from  1  to  5  topics.    Table  3  shows  the  physician 
complements  for  the  18  sites  along  with  the  estimated  patients  served 
and  indication  of  topics  reviewed  at  the  sites.    Titles  for  the  topics 
may  be  found  on  Table  4  which  details  the  review  activity  by  cycle  and 
topic. 

PATIENTS  AND  VISITS  REVIEWED 

Table  4  lists  all  of  the  13  topics  reviewed  during  the  two  cycles. 
No  data  is  presented  for  four  of  the  topics  for  which  the  review  process 
was  not  completed  through  the  intervention  phase.    Review  activity 
ranged  from  Chemistry  Screens,  the  largest,  for  which  24  of  the  31 
physicians  practicing  at  15  sites  and  serving  an  estimated  101,500 
patients  were  subjected  to  reviews  involving  636  patient  charts  to 
Coumadin  for  which  6  of  12  physicians  practicing  at  two  sites  were  sub- 
jected to  reviews  involving  only  four  patient  charts.    Of  the  total  of 
1,427  charts  reviewed  during  the  project,  695  were  reviewed  during  the 
first  cycle  and  732  during  the  second  cycle. 

Table  4  also  shows  the  number  of  physicians  found  to  have  varia- 
tions to  criteria  for  each  topic  and  the  total  variations  found. 
Twenty  of  the  25  physicians  reviewed  during  the  first  cycle  were  found 
to  have  one  or  more  variations  while  only  30  of  the  52  second  cycle 
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physicians  reviewed  were  found  to  have  variations.    Physicians  with 
variations  ranged  from  0%  for  Coumadin  to  100%  for  three  of  the  topics. 
The  mean  rate  was  65%,  the  median  82%.    The  largest  number  of  variations 
were  found  in  reviewing  the  Hypertension  topic  with  258. 
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TABLE  3.    BACKGROUND  DATA: 
PHYSICIANS,  PATIENTS  AND  TOPICS  BY  SITE 
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TABLE  4.    REVIEW  DATA: 


PHYSICIANS  AND  PATIENTS  REVIEWED  AND  VARIATIONS  NOTED  BY  TOPIC 
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5 

14 

39,000 

4  (29%) 

180 

2  (50%) 

88 

TOTAL 

22 

65 

184,900 

25  (38%) 

695 

20  (80%) 

373 

1 

Hypoglycemia 

1 

1 

4,000 

1(100%) 

37 

1(100%) 

36 

2 

Diabetes  Mellitus 

3 

16 

48,500 

11  (69%) 

34 

3  (27%) 

64 

3 

Iron  Def.  Anemia 

3 

16 

48,500 

10  (63%) 

21 

3  (30%) 

14 

4 

Chemistry  Screens 

15 

31 

101,500 

24  (77%) 

636 

23  (96%) 

93 

5 

Coumadin 

2 

12 

35,000 

6  (50%) 

4 

0  (0%) 

0 

6 

Lithium  Treatment 

NA 

7 

Psych.  Consultations 

NA 

TOTAL 

24 

76 

237,500 

52  (68%) 

732 

30  (58%) 

207 

GRAND  TOTAL 

46 

141 

422,400 

77  (55%) 

1 ,427 

50  (65%) 

580 

INDIVIDUALS 

18 

34 

108,500 

34(100%) 

*Since  the  final  data  collection  date,  MFMC  has  added  three  more  topics  to  their  second  round  of  review. 
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METHOD 

A  five  percent  random  sample  of  abstracts  was  requested  by  HCMS 
for  the  purpose  of  analyzing  the  availability  of  the  review  data.  Of 
interest  was  the  frequency  with  which  information,  positive  and  negative, 
concerning  each  criteria  was  recorded  on  the  abstract.    The  project 
responded  with  somewhat  larger  samples,  especially  for  those  topics 
involving  a  small  study  population.    HCMS  staff  used  all  of  the  avail- 
able data  in  estimating  review  data  parameters. 

The  importance  of  any  given  data  item  to  the  monitoring  of  the 
quality  of  medical  care  is  related  in  a  positive  direction  to  the 
product  of  three  ratios: 

1.  Probability  of  the  availability  of  the  data  in  the  chart. 

2.  Ratio  of  identified  deficiencies  to  total  items  recorded. 

3.  Relative  importance  of  a  given  deficiency  to  the  care  of 
the  patient. 

The  first  of  the  ratios  has  been  approximated  for  review  data  by 
criteria  category  and  topic  by  using  the  HCMS  sample  of  review  abstracts. 
The  second  of  the  ratios  will  be  discussed  in  connection  with  the  section 
on  Intervention  Data.    The  third  is  a  matter  of  medical  research  and 
opinion  and  is  beyond  the  purview  of  this  report. 

Review  and  intervention  data  through  August  31,  1978  was  submitted 
by  the  report  for  the  total  study  population  and  is  included  in  this 
report.    (Review  and  intervention  activity  continued  beyond  that  date.) 
The  failure  to  meet  a  given  criteria  was  noted  on  the  abstract  as  a 
"variation."    Some  of  these  variations  were  pointed  out  to  the  providers 
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involved  during  the  intervention  process  and  where  the  provider  felt 
that  the  case  had  been  appropriate,  the  term  "justified"  was  used. 
Variations  not  so  justified  were  termed  "deficiencies." 

CRITERIA/DATA  ITEMS 

Table  5  lists  abstracted  data  items  associated  with  criteria  for 
the  eleven  topics  studied.    Organization  is  by  topic  for  which  are 
shown: 

1.  Number  of  cases  studied 

2.  Number  of  cases  in  HCMS  sample 

3.  Mean  years  of  care  for  patients  in  the  sample 

4.  Mean  visits  per  year  for  those  patients 

Data  items  are  listed  for  each  topic  and  appear  in  the  following 
categories: 

A.  History  and  Physical  Exam  Items 

B.  Diagnostic  Procedures  (including  Immunizations) 

C.  Therapy 

D.  Contraindicated  Therapy 

E.  Management  (other  than  therapeutic) 

F.  Outcomes 

Multnomah  criteria  involve  the  interpretation  of  one  or  more  data 
items  in  terms  of  values,  frequencies,  presence  or  absence.  Although 
some  values  are  entered  on  the  abstract  form,  most  entries  indicate  only 
compliance  (+)  or  noncompliance  (-)  with  the  criteria  or  the  fact  that 
the  item  is  not  mentioned  in  the  record  (0).    Criteria  items  are  listed 
within  category  and  the  required  form  of  entry  indicated.    The  number  of 
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patients  for  whom  a  data  item  was  noted  one  or  more  times  is  shown  for 
each  data  item  as  is  the  percent  of  patients  in  the  sample.    Shown  also 
are  the  total  number  of  times  the  item  was  noted  in  the  sample  (#  Proc) 
and  corresponding  rates  per  visit  and  per  patient  year  of  care.  Details 
of  patient  follow  up  are  given  for  Topic  6--Chemistry  Screens  (Table  5H) 


59 


TABLE  5A    CRITERIA/DATA  ITEMS  -  FIRST  CYCLE 


1.  HYPERTENSION 


Total  Cases:  440 
Sample  Size:  31 


Mean  Years  of  Care:  1.52 
Visits  Per  Patient  Year:  5.3 


Criteria  for:    Management  of  Hypertension  in  adults  18  years  of  age  and 
older. 

Perceived  Problem:    Under-treatment  for  essential  hypertension. 

Time  Frame:    Patients  new  to  the  practice  or  diagnosed  after  January  1,  1975. 


CRITERIA/DATA  ITEMS 


Number  of 
Patients  % 


SAMPLE  DATA 

(249) 

Number  of  Per 
Procedures  Visit 


Per 

Patient  Year 


A.  History  and  Physical 

1.  Current  Drugs* 

2.  Blood  Pressure** 

B.  Diagnostic  Procedures 

1.  UA  Complete 

2.  Potassium 

3.  BUN-Creatinine 

4.  Uric  Acid 

5.  EKG 

6.  Chest  X-Ray 

C.  Therapy 

1.    Hypertensive  Drugs' 

D.  Outcome 

1.  Problem+ 

2.  Consultation 


16 
31 


25 
26 
24 
25 
25 
21 


30 


16 

0 


52 
100 


81 
84 
77 
81 
81 
68 


97 


52 
0 


16 
235 


36 
39 
35 
36 
34 
27 


214 


28 
0 


.06 
.94 


14 
16 
14 
14 
14 
11 


86 


.11 

,00 


0.3 
5.0 


0.8 
0.8 
0.7 
0.8 
0.7 
0.6 


4.6 


0.6 
0.0 


*Name  of  drug  recorded 
**Values  recorded 

+Brief  description  of  problem  recorded 
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TABLE  5B    CRITERIA/DATA  ITEMS  -  FIRST  CYCLE 


!.  HYPERACTIVITY 


Total  Cases:  6 
Sample  Size:  2 


Mean  Years  of  Care:  3.33 
Visits  Per  Patient  Year:  5.9 


Criteria  for:    Management  of  Hyperactivity  and  Learning  Disabilities,  children 
under  12  years  of  age. 

Perceived  Problem:    Overuse  of  drugs  for  behavior  control  without  adequate  workup 

and  follow  up 

Time  Frame:    Initial  diagnosis  must  be  made  at  least  six  months  prior  to 
abstraction 

SAMPLE  DATA 


Number  of 
CRITERIA/DATA  ITEMS        Patients  % 


(39) 

Number  of       Per  Per 
Procedures     Visit        Patient  Year 


A.  History  and  Physical 

1 .  Second  Observation 

2.  Early  Onset 

B.  Diagnostic  Procedures 


2 
2 


1.    Psychological  Evaluation  1 


2.    I.Q.  Test 

C.  Therapy 

1.  Psychotherapy 

2.  Tutoring 

3.  Drug  Therapy 

D.  Contraindicated  Therapy 
1.  Tranquilizers 

E.  Management 

1.  Problem 

2.  Referral 


1 


1 

2 
2 


100 
100 


50 
50 


50 
100 
100 


2 
5 


r 
i 


i 

2 
37 


0 
0 


0 
0 


0 
0 


.05 
.13 


.03 
.03 


.03 
,05 
.95 


.00 


.00 
.00 


3 

8 


.2 
.3 
5.6 


0 
0 
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TABLE  5C    CRITERIA/DATA  ITEMS  -  FIRST  CYCLE 


5.   PELVIC  EXAM 


Total  Cases:  69 
Sample  Size:  4 


Mean  Years  of  Care:  MA 
Visits  Per  Patient  Year:  NA 


Criteria  for:    Completion  of  Pelvic  Exam  with  Physical  Exam,  females,  18 
years  of  age  and  older. 


Perceived  Problem: 
Time  Frame: 


Failure  to  complete  a  Pelvic  and  Rectal  Exam,  and  pap 
smear  as  part  of  a  general  physical  exam. 

Exams  after  January  1,  1976. 


CRITERIA/DATA  ITEMS 


Number  of 
Patients  % 


SAMPLE  DATA 

(4) 

Number  of  *  Per 
Procedures  Visit 


Per 

Patient  Year 


A.  History  and  Physical 

1.  Oral  Contraceptives 

2.  Estrogen  Therapy 

3.  Presence  of  Cervix 

B.  Diagnostic  Procedures 

1..    Pelvic  and  Bi -manual 
Exam 

2.  Digital  Rectal  Exam 

3.  Pap  Test 


0 
0 
4 


0 
0 

100 


3  75 
2  50 

4  100 


0 
0 
4 


3 
2 
4 


0 
0 

1.00 


.75 
.50 
1.00 


NA 


*0ne  visit  recorded  for  each  patient. 
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TABLE  5D    CRITERIA/DATA  ITEMS  -  FIRST  CYCLE 


6.   WELL  CHILD  Total  Cases:    180  Mean  Years  of  Care:  0.93 

Sample  Size:       7  Visits  Per  Patient  Year:  3.7 

Criteria  for:  Management  of  Well  Child  Care,  under  six  years  of  age. 


Perceived  Problem:  Differences  in  immunizations  and  exams  between  family 

practitioners  and  pediatricians. 

Time  Frame:  Patient  must  have  been  followed  for  at  least  3  well 
child  visit. 

SAMPLE  DATA 

(24) 

Number  of  Number  of       Per  Per 
 CRITERIA/DATA  ITEMS        Patients     %  Procedures     Visit        Patient  Year 


A.    History  and  Physical 


1.  Height 

7 

100 

24 

1.00 

3.7 

2.  Weight 

7 

100 

24 

1.00 

3.7 

3.    Head  Circ. 

7 

100 

20 

.83 

3.1 

4.    Hx  and  Pe 

7 

100 

24 

1.00 

3.7 

Diagnostic  Procedures 

(Incl.  Immun.) 

1.  Hct 

5 

86 

6 

.25 

0.9 

2.    Urine  Analysis 

0 

0 

0 

.00 

.0 

3.  Vision 

0 

0 

0 

.00 

.0 

4.  DPT 

7 

100 

21 

.88 

3.2 

5.  OPV 

7 

100 

21 

.88 

3.2 

6.  PKU 

7 

100 

7 

.29 

1.1 

7.  Measles 

3 

43 

3 

.13 

0.5 

8.  Mumps 

3 

43 

3 

.13 

0.5 

9.  Rubella 

3 

43 

3 

.13 

0.5 

10.    PPD  or  Tine 

5 

71 

5 

.21 

0.8 

11.  Problem 

0 

0 

0 

.00 

.0 
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TABLE   5E  CRITERIA/DATA  ITEMS  -  SECOND  CYCLE 


1.  HYPOGLYCEMIA 


Total  Cases:  37 
Sample  Size:  3 


Mean  Years  of  Care:  0.81 
Visits  Per  Patient  Year:  4.1 


Criteria  for:    Reactive  (functional)  Hypoglycemia 


Any  Age 


Perceived  Problem:    Inappropriate  Diagnosis 


Time  Frame:    Not  Stated 


SAMPLE  DATA 


CRITERIA/DATA  ITEMS 


Number  of 
Patients  % 


Number  of  Per 
Procedures  Visit 


Per 

Patient  Year 


0 
0 
0 
2 
2 


3 
3 


A.  History  and  Physical 

1.  Perspiration 

2.  Palpitations 

3.  Weakness 

4.  Headaches 

5.  Confusion 

B.  Diagnostic  Procedures 

1.  Plasma  Glucose** 

2.  Five  Hour  GTT** 

C.  Therapy 

1.    Diet  3 

D.  Contraindicated  Therapy 

1.  Adrenal  Cortical  Hormone  0 

2.  Carb.  Diet  0 

E.  Management 

1.    Consultation  0 


0 
0 
0 
67 
67 


100 
100 


100 


0 
0 


0 
0 
0 
5 
2 


8 
3 


0 
0 


.00 
,00 
,00 
,50 
,20 


,80 
,30 


,30 


,00 
,00 


,00 


.0 
.0 
.0 
2.1 
.8 


3.3 
1.2 


1.2 


.0 
.0 


**Values  Recorded 
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TABLE  5F  CRITERIA/DATA  ITEMS  -  SECOND  CYCLE 


DIABETES  MELLITUS 


Total  Cases:  34 
Sample  Size:  15 


Mean  Years  of  Care:  NA 
Visits  Per  Patient  Year:  NA 


Criteria  for:  Diabetes  Mellitus  (Insulin  Dependent)    Any  Age 
Perceived  Problem:  Inappropriate  therapy  -  including  patient  education 
Time  Frame:  Not  Specified 

SAMPLE  DATA 


Number  of 


PRTTFRTA/DATA  TTFMS 

P a  f  "i  pnl-  c 

rdl  I  ell  lo 

% 

10 

A. 

History  and  Physical 

None 

B. 

Diagnostic  Procedures 

1.    Fasting  Plasma  Sugar 

0 

0 

2.    Blood  Sugar  -  Post 

Prandial** 

6 

40 

C. 

Therapy 

1.  Insulin 

15 

100 

2.  Diet 

15 

100 

3.    Patient  Education 

14 

93 

D. 

Contraindicated  Therapy 

1.    Oral  Hypoglycemics 

0 

0 

E. 

Management 

1.    Office  Visit  every  3 

Months ++ 

NA 

2.  Height 

1 

7 

3.  Weight 

15 

100 

F. 

Outcome 

1.    Children  UA 

1 

7 

2.    Adults  UA 

13 

87 

3.    Fasting  Plasma  Glue.** 

3 

20 

4.    Blood  Sugar  -  2  Hours** 

P.P. 

13 

87 

5.    Insulin  Reaction 

5 

33 

6.  Ketoacidosis 

0 

0 

Number  of 
Procedures 


Per 
Visit 


Per 

Patient  Year 


**Values  Recorded 
++Dates  Recorded 


(Two  different  abstract  forms  were 
used;  coult  not  combine  to  complete 
patient-prof iles) 
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TABLE  5G    CRITERIA/DATA  ITEMS  -  SECOND  CYCLE 


ANEMIA 


Total  Cases:  21 
Sample  Size:  7 


Mean  Years  of  Care:  1.61 
Visits  Per  Patient  Year:  4.3 


Criteria  for:    Iron  Deficiency  Anemia 


Age  20  or  over 


Perceived  Problem:   Therapy  with  iron  alone  or  in  combination  without 

minimal  study. 

Time  Frame:    Not  Stated 


SAMPLE  DATA 


CRITERIA/DATA  ITEMS 

Number  of 
Patients 

% 

Number  of 
Procedures 

Per 
Visit 

Per 

Patient  Year 

A. 

History  and  Physical 

1. 

Non-Gastro.  Bleeding 

0 

0 

0 

.00 

.0 

B. 

Diagnostic  Procedures 

1. 

HCT** 

7 

100 

22 

.46 

2.0 

2. 

Hemo. ** 

7 

100 

26 

.54 

2.3 

3. 

CBC-MCV 

0 

0 

0 

.00 

.0 

4. 

CBC-MCHC  + 

1 

14 

3 

.06 

0.3 

5. 

CBC-MCC 

0 

0 

0 

.00 

.0 

6. 

Hypochromic  Blood 

Smear 

1 

14 

3 

.06 

0.3 

7. 

Upper  GI  or  Gastroscopy 

1 

14 

1 

.02 

0.1 

8. 

Barium  Enema 

0 

0 

0 

.00 

.0 

9. 

Coloscopy  or  Sigmoid. 

2 

29 

5 

.10 

0.4 

C. 

Therapy 

1. 

Iron  Therapy** 

7 

100 

41 

.85 

3.6 

D. 

Management 

1. 

Consultation 

1 

14 

1 

.02 

0.1 

**Values  Recorded 
+Brief  description  of  problem  recorded 
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TABLE  5H  CRITERIA/DATA  ITEMS  -  SECOND  CYCLE 


CHEMISTRY  SCREENS         Total  Cases:    636  Mean  Years  of  Care:  NA 

Sample  Size:     57  Visits  Per  Patient  Year:  NA 

Criteria  for:    Chemistry  Screen  Results 

Perceived  Problem:    Inadequate  follow-up  for  abnormal  chemistry 

screen  results 

Time  Frame:    Not  Specified. 


CRITERIA/DATA  ITEMS  SAMPLE  DATA 


Patients 

Patients 

Patients 

With 

Test 

Abnormal s 

With 

Retest 

Abnormal s 

With 

Notatio 

Diagnostic  Procedures 

#_ 

% 

# 

% 

# 

% 

# 

% 

# 

% 

1.  K+ 

50 

88 

10 

18 

3 

5 

2 

4 

7 

12 

2.  Bilirubin 

56 

98 

15 

26 

5 

9 

4 

7 

8 

14 

3.  Creatinine 

51 

89 

5 

9 

5 

9 

3 

5 

1 

2 

4.  Calcium 

57 

100 

9 

16 

1 

2 

0 

0 

6 

11 

5.    Uric  Acid 

57 

100 

15 

26 

5 

9 

0 

0 

9 

16 

6.  S60T 

57 

100 

6 

11 

4 

7 

3 

5 

2 

4 

7.  Protein 

46 

98 

5 

9 

3 

5 

0 

0 

1 

2 

8.    Random  Bid.  Glue. 

57 

100 

24 

42 

3 

5 

1 

2 

15 

26 

♦Explanatory  Notation. 
Values  required  for  all  procedures. 
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TABLE  51    CRITERIA/DATA  ITEMS  -  SECOND  CYCLE 


COUMADIN 


Total  Cases:  4+++ 
Sample  Size:  5 


Mean  Years  of  Care:  0.72 
Visits  Per  Patient  Year:  16.2 


Criteria  for:    Use  of  Coumadin 


Age:    All  ages 


Perceived  Problem: 


Anticoagulant  therapy  (Coumadin)  without  knowledge  of 
coagulability  of  the  patient's  blood,  i.e.,  treatment 
without  monitoring  the  prothrombin  time. 


Time  Frame:    Not  Specified 


SAMPLE  DATA 


Number  of  Number  of      Per  Per 
 CRITERIA/DATA  ITEMS        Patients     %  Procedures     Visit        Patient  Year 

A.  History  and  Physical 

1.  Thrombophlebitis  2  40  2  .03  0.6 

2.  CVA  0  0  0  .00  .0 

3.  MI  1  20  1  .02  0.3 

4.  Pulmonary  Embolus  1  20  1  .02  0.3 

5.  Prosth.  Heart  Valve  1  20  1  .02  0.3 

B.  Diagnostic  Procedures 

1.    Protime**  5  100  55  .95  15.4 

C.  Therapy 

1.    Coumadin^  5  100  51  .88  14.2 

D.  Contraindicated  Therapy 

1.  Hemorrhage  0  0  0  .00  .0 

2.  Pregnancy  0  0  0  .00  .0 

F.  Outcome 

1.    Symptoms+  3  60  5  .09  1.4 

^Dosage 
**Values 
+++Count  reported  in  letter 
+Brief  description  of  problem  recorded 
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AVAILABILITY  OF  DATA 

The  importance  of  any  given  data  item  to  the  monitoring  of  the 
quality  of  medical  care  is  related  in  a  positive  direction  to  the 
product  of  three  ratios: 

1.  Probability  of  the  availability  of  the  data  in  the  chart 

2.  Ratio  of  identified  deficiencies  to  total  items  recorded 

3.  Relative  importance  of  a  given  deficiency  to  the  care  of  the 
patient 

The  first  of  the  ratios  has  been  approximated  for  review  data  by 
criteria  category  and  topic  by  using  the  HCMS  sample  of  review  abstracts. 
The  second  of  the  ratios  will  be  discussed  in  connection  with  the  section 
on  Intervention  Data.    The  third  is  a  matter  of  medical  research  and 
opinion  and  is  beyond  the  purview  of  this  report. 

Table  6  details  the  dimension  of  the  HCMS  Sample  of  Review  Data. 
The  sample  included  131  abstracts  among  the  9  topics  for  which  data  was 
available  and  represented  about  10%  of  the  total  cases  reviewed. 

Table  7  shows  the  percentage  of  availability  of  data  for  reviewing 
97  criteria  data  items  classified  among  6  categories  and  distributed 
among  the  9  topics.    The  number  of  cases  in  the  sample  is  listed  for 
each  topic  (S).    Numbers  of  data  items  (C),  patients  for  whom  such  data 
items  are  recorded  (N),  are  listed  by  topic  and  category.    The  percent- 
age of  availability  (%)  is  calculated  by  this  formula: 

%  =  100  N/(S  ■  C) 

It  may  be  noted  that  while  data  items  in  the  Diagnostic  and  Therapy 
categories  were  above  average  in  availability  and  History  and  Physical 
items  were  near  the  average  for  all  categories,  Management  and  Outcome 
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categories  showed  very  low  degrees  of  availability.    Data  items  for  the 
Contraindicated  Therapy  category  were  not  recorded  in  the  sample  at  all. 
While  the  ratio  of  variations  to  items  recorded  may  be  expected  to  be 
high  for  Contraindicated  Therapies,  the  extremely  low  data  availability 
ratio  for  this  group  of  data  items  would  suggest  that  there  might  be 
few,  if  any,  variations  expected. 
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TABLE  6. 

DISTRIBUTION 

OF  CASES 

REVIEWED  AND  HCMS 

SAMPLE  BY 

TOPIC 

TOPIC 

Sites 
Reviewed 

Total 

Physicians 
at  Sites 

Physicians 
Reviewed 

Estim. 
Patients 

Served 
by  Sites 

Cases 
Reviewed 

Cases  in 

HCMS 
Sample  (%) 

I  1. 

Hypertension 

11 

27 

17  (63%) 

89,400 

440 

31  (7) 

2. 

Hyperactivity 

5 

14 

2  (14%) 

39,000 

6 

2  (33) 

5. 

Pelvic  Exam 

2 

12 

2  (17%) 

33,500 

69 

4  (6) 

6. 

Well  Child  Exam 

3 

12 

4  (33%) 

33,000 

180 

7  (4) 

II  1. 

Hypoglycemia 

1 

1 

1(100%) 

4,000 

37 

3  (8) 

2. 

Diabetes  Mellitus 

3 

16 

11  (69%) 

48,500 

34 

15  (44) 

3. 

Anemi  a 

3 

1  0 

IU  (bo/o) 

zip  nnn 
t-o ,  uuu 

c  \ 

7  (W\ 

4. 

Chemistry  Screens 

15 

31 

24  (77%) 

101 ,500 

636 

57  (9) 

5. 

Coumadi  n 

9 

1  c. 

fi   ( RD0/ ) 

U     \*J\J/o  / 

35  nnn 

4 

5(125) 

TOTAL 

45 

141 

72  (51%) 

431 ,900 

1,427 

131  (10) 

INDIVIDUALS 

18 

34 

34(100%) 

108,500 

TABLE  7.  AVAILABILITY  OF  REVIEW  DATA  BY  TOPIC 
AND  CRITERIA  CATEGORY  BASED  ON  HCMS  SAMPLE  DATA 


History        Diagnostic  Contraindicated 
^  and  Physical     Procedures    Therapy         Therapy  Management     Outcome  TOTAL 

TOPIC   S        C       N       %       C     N     %     C     N     %     C       N       %       C     N     %     C     N     %     C       N  % 


I    1.  Hypertension 

31 

2 

47 

76 

6 

146 

78 

1 

30 

97 

2 

16 

28 

11 

239 

70 

2.  Hyperactivity 

2 

2 

4 

100 

2 

2 

50 

3 

5 

83 

1 

0 

0 

2 

0 

0 

10 

11 

55 

5.  Pelvic  Exam 

4 

3 

A 

4 

33 

3 

y 

10 

6 

13 

54 

6.  Well  Child  Exam 

7 

4 

28 

1 00 

1  1 

A  1 

4 1 

b3 

15 

69 

66 

TOTAL 

A  A 

44 

1  1 

83 

78 

/  1 

A 

3D 

yo 

1 

u 

n 
u 

c 

n 

u 

n 

2 

Cm 

16 

28 

42 

332 

68 

II    1.  Hypoglycemia 

3 

5 

4 

27 

2 

6 

100 

1 

3 

100 

2 

0 

0 

1 

0 

0 

11 

13 

39 

2.  Diabetes  Mellitus 

15 

2 

6 

20 

3 

44 

98 

1 

0 

0 

2 

16 

53 

6 

35 

39 

14 

101 

48 

3.  Iron  Deficiency 
Anemia 

7 

1 

0 

0 

9 

19 

30 

1 

7 

100 

1 

1 

14 

12 

27 

32 

4.  Chemistry  Screens 

57 

8 

441 

96 

8 

441 

96 

5.  Coumadin 

_5 

_5 

_5 

20 

_2 

5 

100 

J_ 

_5 

100 

_2 

_g 

_g 

J_ 

_3 

60 

10. 

18 

36 

TOTAL 

87 

11 

9 

19 

22 

477 

85 

6 

59 

98 

5 

0 

0 

4 

17 

43 

7 

38 

40 

55 

600 

72 

GRAND  TOTAL 

131 

22 

92 

60 

44 

675 

80 

10 

94 

97 

6 

0 

0 

6 

17 

39 

9 

54 

34 

97 

932 

70 

Key:    S  -  Sample  Size 

C  -  Number  of  data  items  listed  as  criteria  for  this  category 

N  -  Numerator,  number  of  patients  for  which  each  data  item  was  recorded,  summed  by  category 

%  -  100  N/(S  •  C) 


REVIEW  AND  INTERVENTION 

Table  8  presents  for  each  topic  a  quantitative  description  of  the 
review  activity  associated  with  that  topic.    Data  is  classified  on  the 
basis  of  the  six  categories  of  criteria  described  in  the  preceding 
section.    The  table  lists  the  total  number  of  criteria  for  each  category, 
the  number  of  physicians  for  whom  variations  were  recorded  and  the  total 
of  those  variations. 

Table  9  summarizes  the  variation  data  detailed  in  Table  8  by  criteria 
category  and  cycle.    As  previous  studies  have  indicated,  use  of  essential 
criteria  can  result  in  large  variation  rates.    Therefore,  until  the  justifi 
cation  procedures  are  complete,  the  variation  rate  per  physician  should 
not  be  interpreted  as  a  reflection  of  quality  of  care.    It  is  quite 
evident  that  diagnostic  procedure  criteria  are  not  only  the  most  used  in 
review  but  also  by  far  the  most  productive  in  identifying  variations. 
Criteria  in  the  History  and  Physical  and  the  Therapy  categories  were 
more  productive  of  variations  for  topics  in  the  first  cycle  while  Manage- 
ment and  Outcome  criteria  were  more  productive  for  second  cycle  topics. 
Few  Contraindicative  criteria  were  used  and  these  produced  no  variations 
in  any  of  the  studies. 

Comparing  the  results  of  Table  9  with  the  percentage  of  data 
availability  developed  for  the  HCMS  sample  in  Table  7  gives  some  insight 
into  criteria  productivity.    In  the  History  and  Physical  category  data 
was  78%  available  on  first  cycle  topics  but  only  19%  available  on  second 
cycle.    The  difference  between  first  and  second  cycle  productivity  of 
variations  may  be  explained  by  the  difference  in  data  availability. 
Indeed  referring  back  to  the  criteria/data  item  listings  in  Table  5  we 
note  that  the  first  cycle  topics  used  many  readily  available  items  such 

73 


as  blood  pressure,  height  and  weight  while  the  second  cycle  criteria 
involved  symptoms  and  histories  of  symptoms.    On  the  other  hand  criteria 
in  the  Therapy  category  had  high  data  availability  levels  in  both  cycles, 
but  produced  relatively  few  variations  in  the  second  cycle.  Contra- 
indicated  Therapies  were  not  observed  in  the  HCMS  sample  of  review  data 
and  the  absence  of  variations  may  indicate  that  none  were  observed  in 
the  studies.    Since  complete  data  sets  were  not  available  on  justifica- 
tion rates  for  variations  reported  here,  no  justification  statistics  are 
shown  in  the  previous  tables. 
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TABLE  8A.    VARIATION  DATA 
II:  Hypertension 


No.  Of 

Phys.  With 

No.  Of 

Criteria  Category 

Crit. 

Variations 

Variations 

A. 

History  &  Physical 

2 

2 

4 

B. 

Diagnostic  Procedure 

(Incl.  Immun.) 

6 

13 

212 

C. 

Therapy 

1 

12 

42 

D. 

Contraindications 

E. 

Management 

F. 

Outcome 

2 

TOTAI 

11 

27 

coo 

TABLE  8B 

.    VARIATION  DATA 

T?  • 

l  L  . 

Hyperactivity 

No.  Of 

Phys.  With 

No.  Of 

Criteria  Category 

Crit. 

Variations 

Variations 

A. 

History  &  Physical 

2 

2 

4 

B. 

Diagnostic  Procedure 

(Incl.  Immun.) 

2 

2 

9 

C. 

Therapy 

3 

D. 

Contraindications 

1 

E. 

Management 

2 

F. 

Outcome 

TOTAL 

10 

4 

13 
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TABLE  8C.    VARIATION  DATA 
15:    Pelvic  Exam 


No.  Of    Phys.  With     No.  Of 
Criteria  Category  Crit.     Variations  Variations 

A.  History  &  Physical  3  1  3 

B.  Diagnostic  Procedure 

(Incl.  Immun.)  3  2  11 

C.  Therapy 

D.  Contraindications 

E.  Management 

F.  Outcome 

TOTAL  6  3  14 


TABLE  8D.    VARIATION  DATA 
16.    Well  Child  Exam 


No.  Of    Phys.  With     No.  Of 
Criteria  Category  Crit.     Variations  Variations 

A.  History  &  Physical  4  2  47 

B.  Diagnostic  Procedure 

(Incl.  Immun.)  11  2  41 

C.  Therapy 

D.  Contraindications 

E.  Management 

F.  Outcome 

TOTAL  15  4  88 
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TABLE  8E.    VARIATION  DATA 
III:  Hypoglycemia 


No.  Of 

Phys.  With 

No.  Of 

Criteria  Category 

Crit. 

Variations 

Variations 

A 

Hi^tnrv  &  Phv^iral 

l 
i 

? 

B. 

Diagnostic  Procedure 

(Incl.  Immun.) 

2 

1 

19 

C. 

Therapy 

1 

D. 

Contraindications 

2 

E. 

Management 

1 

1 

15 

F. 

Outcome 

TOTAL 

11 

3 

36 

TAni  r 

IAdLl 

8F.    VARIATION  DATA 

• 

112: 

Diabetes 

Mel  1 itus 

No.  Of 

Phys.  With 

No.  Of 

Criteria  Category 

Crit. 

Variations 

Variations 

A. 

History  &  Physical 

B. 

Diagnostic  Procedure 

(Incl.  Immun.) 

2 

C. 

Therapy 

3 

1 

3 

D. 

Contraindications 

1 

E. 

Management 

3 

3 

30 

F. 

Outcome 

6 

3 

31 

TOTAL 

15 

7 

64 
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TABLE  8G.    VARIATION  DATA 
113:  Anemia 


No.  Of   Phys.  With     No.  Of 
Criteria  Category  Crit.     Variations  Variations 


A. 

History  &  Physical 

1 

B. 

Diagnostic  Procedure 

(Inch  Immun.) 

9 

2 

11 

C. 

Therapy 

1 

2 

3 

D. 

Contraindications 

E. 

Management 

1 

F. 

Outcome 

TOTAL 

12 

4 

14 

TABLE  8H.    VARIATION  DATA 
114:    Chemistry  Screens 


No.  Of    Phys.  With     No.  Of 
Criteria  Category  Crit.     Variations  Variations 

A.  History  &  Physical 

B.  Diagnostic  Procedure 

(Incl.  Immun.)  8  14  93 

C.  Therapy 

D.  Contraindications 

E.  Management 

F.  Outcome 

TOTAL  8  14  93 
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TABLE  81.    VARIATION  DATA 
115:  Coumadin 


No.  Of    Phys.  With     No.  Of 
Criteria  Category  Crit.     Variations  Variations 

A.  History  &  Physical  5 

B.  Diagnostic  Procedure 

(Incl.  Immun.)  1 

C.  Therapy  1 

D.  Contraindications  2 

E.  Management 

F.  Outcome  1 

TOTAL  10 
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TABLE  9.    CRITERIA  VARIATIONS  BY  CATEGORY 


FOR  CYCLES  I  AND  II  TOPICS  AND  TOTAL 

Phys.  With  Total 
No.  Of  Crit.    Variations  Variations 


Criteria  Category 

I 

II 

T 

I 

II 

T 

I 

II 

T 

1. 

History  and  Physical 

11 

11 

22 

7 

1 

8 

58 

2 

60 

2. 

Diagnostic  Proc. 

22 

22 

44 

19 

17 

36 

273 

123 

396 

3. 

Therapy 

4 

6 

10 

12 

3 

15 

42 

6 

48 

4. 

Contraindications 

1 

5 

6 

0 

0 

5. 

Management 

2 

5 

7 

4 

4 

45 

45 

6. 

Outcome 

2 

7 

9 

3 

3 

31 

31 

TOTAL 

42 

56 

98 

38 

28 

66 

373 

207 

580 
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SECTION  VI 
RESOURCES  AND  COST 
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INTRODUCTION  AND  METHOD 

This  section  presents  the  personnel  resources  used  by  MFMC  in 
planning,  developing  and  operating  an  ambulatory  review  program  from 
October  1,  1976  to  June  30,  1978. 

Personnel  and  other  resources  for  specific  functions  are  analyzed 
to  present  trends  over  time  and  to  provide  some  indication  of  the 
resources  required  by  review  organizations  during  implementation  of 

ambulatory  review. 

To  develop  accurate  information  on  resource  use,  MFMC  collected 
personnel  hours  spent  on  specific  activities  and  forwarded  the  informa- 
tion to  HCMS.    Three  conditions  suggested  special  procedures  for  record- 
ing and  collecting  personnel  hours: 

1.  Having  MFMC  personnel  record  their  hours  was  likely  to 
increase  the  face  validity  of  the  information  collected. 

2.  Vouchered  cost  information  was  not  considered  reliable 
because  of  the  variety  of  PSRO  procedures  used  in  billing 
the  Health  Standards  and  Quality  Bureau  (HSQB). 

3.  Vouchered  personnel  information  frequently  does  not  con- 
tain volunteer  physician  time  and  overtime  which  are 
required  for  an  accurate  estimate  of  resource  use. 

In  collecting  resource  information  HCMS  and  each  PSRO  discussed 
systematic  recording  procedures,  developed  data  compilation  procedures 
and  defined  specific  terms  and  categories.    Participants  in  the  demon- 
stration project  were  asked  to  describe  their  ambulatory  review  activities 
and  record  the  number  of  hours  spent  on  a  daily  or  weekly  basis.  During 
initial  discussions,  HCMS  and  MFMC  developed  rigorous  procedures  for 
collection  of  personnel  data,  and  a  monthly  activity  log  with  a  standard 
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format  was  designed  to  record  information.    The  logs  and  all  minutes 
of  meetings  were  forwarded  to  HCMS  on  a  monthly  basis. 

Categorization  and  Compilation  of  Resource  Data 

To  facilitate  the  compilation  and  presentation  of  data,  HCMS 
classified  the  information  contained  in  the  MFMC  logs  into  a  standard 
format  composed  of  three  personnel  categories: 

1.  Physicians—including  physicians  working  either  in  staff 
capacity  or  as  special  panel  consultants. 

2.  Management/Technical 

a)  Management—all  personnel  assigned  the  responsibility 
to  administer  the  project  or  to  assist  in  managing 
the  project 

b)  Technical—personnel  principally  engaged  in  technical 
design  of  the  project,  analysis  of  the  data,  providing 
liaison  with  data  processing,  etc. 

3.  Data  Collection/Clerical 

a)  Data  Collection— those  personnel  retrieving  data  from 
medical  record  charts,  transforming  and  coding  data 
for  subsequent  processing,  etc. 

b)  Clerical— secretarial  personnel  whose  duties  are  typing, 
filing  and  generally  supporting  the  project. 

Each  participant  in  the  demonstration  project  was  classified  into  one  o 
more  of  the  above  three  categories  depending  on  type  of  work  performed 
during  the  reporting  periods.    Where  the  choice  between  categories  2  an 
3  was  not  clear,  persons  receiving  $7  or  more  per  hour  were  assigned 
to  the  Management/Technical  category  and  those  receiving  less  than  $7 
per  hour  to  the  Data  Collection/Clerical  category. 

83 


Six  major  functional  categories  were  identified  as  spanning  the 
entire  work  effort  of  the  project.    The  six  functions  are: 

1.  Planning  and  Administration—includes  activities  relating  to 
system-wide  staffing  and  coordination,  system  design  and 
approach  and  the  administrative  support  activities  that 
facilitate  review  system  implementation. 

2.  Topic  Selection—work  related  to  identifying  problems  or 
topics  for  review  and  the  selection  of  those  topics  likely  to 
produce  the  greatest  benefit. 

3.  Criteria  Development—development  of  criteria  related  to  the 
problem  or  topic  under  study  which  can  be  measured  or  compared 
by  data  being  collected. 

4.  Data  Collection  (includes  abstracting)— activity  of  collecting 
information  from  required  data  sources. 

5.  Review  and  Decision  Making— professional  (physician  or  other 
staff)  examination  and  interpretation  of  processed  information. 

6.  Feedback  and  Intervention— providing  to  those  individual/ 
facilities/groups  results  of  analysis  and/or  review  and 
decision  making. 

Assignment  of  personnel  hours  to  specific  functions  was  done  by 
MFMC  and  reported  on  a  monthly  basis  on  the  logs. 

PRESENTATION  OF  RESOURCE  DATA 

The  summary  of  total  personnel  hours  distributed  by  personnel 
category  and  tabulated  by  function  is  presented  in  Table  10.  Percents 
of  personnel  resources  for  each  of  the  functions  is  shown  in  Total  and 
separately  for  each  personnel  category.    Figure  3  is  a  graphic  repre- 
sentation of  this  data.    In  this  graph,  the  percent  of  total  hours 
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worked  by  members  of  each  of  the  three  personnel  categories  is  represented 
horizontally  and  the  percent  of  hours  used  for  each  functional  category 
is  represented  vertically.    Areas  within  the  bars  are  proportional  to 
the  number  of  hours  expended  by  any  one  personnel  category  for  any 
one  function. 
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TABLE  10:    DISTRIBUTION  OF  TOTAL  PERSONNEL  HOURS 
BY  FUNCTION  AND  PERSONNEL  CATEGORY 


PERSONNEL  CATEGORY 


MGMT/ 

DATA/ 

FUNCTION 

MD  (%) 

TECHNICAL  (%) 

CLERICAL  (%) 

TOTAL  (%) 

Planning  and  Administration 

183  (  33) 

742  (  45) 

2,042  (  30) 

2,967  (  33) 

Topic  Selection 

97  (  17) 

255  (  15) 

354  (  5) 

706  (  8) 

Criteria  Development 

205  (  37) 

76  (  5) 

303  (  5) 

584  (  7) 

Data  Collection 

28  (  5) 

346  (  21) 

3,528  (  52) 

3,902  (  43) 

Review  and  Decision  Making 

4  (  1) 

150  (  9) 

124  (  2) 

278  (  3) 

Feedback  and  Intervention 

45  (  8) 

76  (  5) 

437  (  6) 

558  (  6) 

TOTAL 

562  (100) 

1  ,645  (100) 

6,788  (100) 

8,995  (100) 

PERCENT 

6% 

18% 

76% 

100% 
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FIGURE  3 

DISTRIBUTION  OF  TOTAL  PERSONNEL  HOURS  BY  FUNCTION  WITHIN  PERSONNEL  CATEGORY 
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TABLE  11:    DISTRIBUTION  OF  PERSONNEL  HOURS  FOR  FUNCTIONS 
BY  CALENDAR  QUARTER  AND  PERSONNEL  CATEGORY 


PERSONNEL  CATEGORY 


FUNCTION 


Planning  and 
Administration 


Topic  Selec- 
tion 


Criteria 
Development 


Data  Collec- 
tion 


CALENDAR 

MGMT/ 

DATA/ 

QUARTER 

MD  (%) 

TECHNICAL  (%) 

CLERICAL  (%) 

TOTAL  (%) 

10-12/76 

68  (  37) 

206  (  28) 

152  (  7) 

426  (  14) 

1-3  /77 

25  (  14) 

240  (  32) 

528  (  26) 

793  (  27) 

4-6 

13  (  7) 

118  (  16) 

360  (  17) 

491  (  17) 

7-9 

23  (  13) 

112  (  15) 

282  (  14) 

417  (  14) 

10-12 

54  (  29) 

30  (  4) 

240  (  12) 

324  (  11) 

1-3  /78 

36  (  5) 

240  (  12) 

276  (  9) 

4-6 

240  12 

?4f)  (  P,S 

TOTALS 

183  (100) 

742  (100) 

2,042  (100) 

2,967  (100) 

PERCENT 

6% 

25% 

69% 

100% 

10-12/76 

22  (  23) 

36  (  14) 

58  (  8) 

1-3  /77 

30  (  31) 

48  (  19) 

44  (  12) 

122  (  17) 

4-6 

1  (  1) 

10  (  3) 

11  (  2) 

7-9 

5  (  5) 

20  (  8) 

25  (  4) 

10-12 

37  (  38) 

45  (  18) 

142  (  40) 

224  (  32) 

1-3  /78 

48  (  19) 

158  (  45) 

206  (  29) 

2  (  2) 

58  22 

fin  (  p,\ 

TOTAL 

97  (100) 

255  (100) 

354  (100) 

706  (100) 

PERCENT 

14% 

36% 

50% 

100% 

10-12/76 

1-3  /77 

63  (  31) 

22  (  29] 

I           66  (  22) 

151  (  26) 

4-6 

103  (  50) 

60  (  20) 

163  (  28) 

7-9 

39  (  19) 

6  (  8] 

1            10  (  3) 

55  (  9) 

10-12 

1-3  /78 

32  (  42] 

1          167  (  55) 

199  (  34) 

4-6 

16  21 

16  3 

TOTAL 

205  (100) 

76  (loo; 

1          303  (100) 

584  (100) 

PERCENT 

35% 

13% 

52% 

100% 

10-12/76 

43  (  1) 

1-3  /77 

15  (4)            28  (  1) 

4-6 

3  (  11) 

147  (  43)          928  (  26) 

1,078  (  28) 

7-9 

25  (  89) 

94  (  27)          708  (  20) 

827  (  21) 

10-12 

630  (  16) 

1-3  /78 

40  (  12 

)          590  (  17) 

4-6 

50  14 

)       1,274  36 

1,324  34 

TOTAL 

28  (100) 

346  (100 

)       3,528  (100) 

3,902  (100) 

PERCENT 

1% 

9% 

90% 

100% 
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TABLE  11  (CONT'D) 


FUNCTION 


Review  and 
Decision  Making 


CALENDAR 
QUARTER 

10-12/76 

1-3  111 

4-6 

7-9 
10-12 

1-3  /78 

4-6 
TOTAL 
PERCENT 


PERSONNEL  CATEGORY 


MGMT/  DATA/ 
MP    (%)    TECHNICAL  (%)      CLERICAL  (%) 


4 

(100) 

124 

4 

(100) 

150 

26  (  17) 


83 


54% 


18  (  14) 
106  (  86 


124  (100 
45% 


TOTAL  {%) 


44  (  16) 


234 

278 

iiooj 

100% 


Feedback  and 
Intervention 


10-12/76 

1-3  /77 

4-6 

7-9 
10-12 

1-3  /78 

4-6 
TOTAL 
PERCENT 


32  (  71) 

13  (  29 
45  (100 

8% 


46  1 

:  60] 

i          368  1 

:  84] 

1             446  1 

:  80) 

18  1 

!  24i 

I             5  1 

;  V\ 

1              23  1 

;  4) 

12  1 

1           64  1 

1              89  i 

liooj 

76  i 

hooi 

1          437  i 

hoo! 

1             558  i 

14% 


78% 


100% 
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APPENDIX  A 


MULTNOMAH  FOUNDATION  FOR  MEDICAL  CARE 
CRITERIA 
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APPENDIX  A 


MULTNOMAH  FOUNDATION  FOR  MEDICAL  CARE 
CRITERIA 
Study  Cycle  I 


TOPICS 

1.  Hyperactivity 

2.  Well  child  care 

3.  Pediatric  gastroenteritis 

4.  Gonorrhea  culture 

5.  Pelvic  exam 

6.  Hypertension 

CRITERIA 
HISTORY  &  PHYSICAL 

1.  One  or  more  observation 

2.  Chronic  early  onset 

3.  Vomiting  and/or  diarrhea 

4.  Reference  to  dehydration 

5.  Serial  weight 

6.  Leukorrhea 

7.  Condyloma 

8.  Trichomonas 

9.  Herpes  progenitalia 

10.  Pelvic  inflammatory  disease 

11.  Positive  contact  (gonorrhea) 

12.  Use  of  contraceptives 

13.  Use  of  estrogen  therapy 

14.  Presence  of  cervix 

15.  BP > 160/100  two  occasions 

16.  Oral  contraceptive  use 

17.  History  of  use  of  sympathomimetics 

18.  History  of  use  of  cortisone 

DIAGNOSTIC  PROCEDURES 

1.  Psychological  evaluation 

2.  I.Q.  test 

3.  Height 

4.  Weight 

5.  Head  circumference 

6.  History  and  physical 

7.  Hematocrit 

8.  Urinalysis 

9.  Vision  test 

10.  CBC 

1 1 .  Gonorrhea  cul ture 

12.  Pelvic  and  bimanual 

13.  Pap  test 

14.  Digital  rectal  exam 

15.  Potassium  and  BUN 

16.  Uric  acid 

17.  EKG 

18.  Chest  x-ray 
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MFMC  CRITERIA  -  CONTINUED 


CRITERIA 

1 


THERAPY 

1 .  Psychotherapy  x 

2.  Tutoring  x 

3.  Drugs  -  ritalin,  dexedrine,  cylert  x 

4.  DPT 

5.  OPV 

6.  PKU 

7.  Measles  vaccine 

8.  Mumps  vaccine 

9.  Rubella  vaccine 
10.  PPD  or  tine 

n.  npo 

12.  Clear  liquids 

13.  Ampicillin,  penicillin,  spectenomycin 

14.  Tetracycline  if  allergic  penicillin 

15.  Four  phases  drug  therapy 

CONTRAINDICATED  THERAPY 

1.  Tranquilizers  x 

2.  Antidiarrheals 

3.  Antiemetics 

4.  Antibiotics 

OUTCOME 

1.  Cessation  of  yomiting/diarrhea 

2.  Negative  repeat  culture  14  days 

3.  Lower  BP  toCI 40/90  in  3  months 

PATIENT  DISPOSITION 

1 .  Fol low-up  visit  x 

2.  Referral  x 

3.  Problem 


ADMINISTRATIVE  DATA  ELEMENTS  --  ALL  TOPICS 

1.  Physician  identification 

2.  Physician  specialty 

3.  Patient  identification 

4.  Birthdate 

5.  Age 

6.  Sex 

7.  Payment  code 

8.  Date  of  visit 

9.  Category  of  visit 
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MULTNOMAH  FOUNDATION  FOR  MEDICAL  CARE 

CRITERIA 
Study  Cycle  II 

TOPICS 

1.  Reactive  hypoglycemia 

2.  Diabetes  nielli tus 

3.  Iron  deficiency  anemia 

4.  Use  of  Coumadin 

5.  Lithium  therapy  for  bipolar  depression  and  mania 

6.  Psychiatric  consultations 

7.  Chemistry  screen  results  (see  separate  sheet  for  criteria) 

TOPIC 

CRITERIA  J_  2^      3        4        5  6 

HISTORY  &  PHYSICAL 

1.  Perspiration,  palpitation,  weakness 
relieved  by  ingestion  carbohydrates  x 

2.  Never  present  in  fasting  state  x 

3.  Previous  thrombophlebitis,  CVA,  MI, 
pulmonary  embolus  or  prosthetic  heart 
valves 

4.  Cyclical  mood  swings 

5.  Periods  of  mania  or  hypomania 

6.  No  major  predisposing  life  events 

7.  Stated  reason  for  consultation 

8.  Reason  for  consult  responded  to 

DIAGNOSTIC  PROCEDURES 

1.  Plasma  glucose  50  or  less  x 

2.  GTT  50  or  less  x 

3.  Confirmed  FBS  150  mgm  or  more 

4.  Two  hour  PPBS  >180  mgm  or  150  mgm 
at  three  hours 

5.  Hb<ll  females, <12  males 

6.  Hct<35  females, < 40  males 

7.  CBC:    MCV<80,  or  MCHC<27,  or 
MCC<30 

8.  Hypochromic  blood  smear 

9.  Prothrombin  min.  once  per  month 

10.  Normal  physical  exam 

11.  Blood  lithium  levels 


x 

X 
X 


X 
X 


X 
X 


X 
X 


X 
X 
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MFMC  CRITERIA  (Cont'd) 


CRITERIA 


THERAPY 

1.  Diet  prescribed  x 

2.  Insulin 

3.  Patient  ed:    insulin,  treatment 
of  reactions,  care  during  illness 

4.  Lithium  with  lithium  blood  levels 

CONTRAINDICATED  THERAPY 

1.  Adrenal  cortical  hormone  x 

2.  Carbohydrate  diet<50  grm/day  x 

3.  Oral  hypoglycemics 

4.  ASA,  Atromid-S,  Butazolidine 

5.  Evidence  of  hemorrhage  or  ecchymoses 

6.  MOA,  antidepresssants 

7.  Diuretics 

SHORT  TERM  OUTCOME 

1.  Controlled  within  1  month  of  insulin 
initiation 

2.  No  evidence  of  ecchymoses,  hematuria, 
etc. 

3.  Lithium  levels  twice  weekly  until 
stable 

INVESTIGATIVE  PROCEDURES 

1.  Upper  GI 

2.  Barium  enema  studies 

3.  Sigmoidoscopy 

LONG  TERM  MANAGEMENT/OUTCOME 

1.  Office  visit  min.  every  3  mo. 

2.  Documented  control  min.  2/3  FU  visits 

3.  Documentation  of  height  (<18  yr)  and 
weight 

4.  Prevention  of  thrombo-embol i  with 
Coumadin 

5.  Lithium  blood  levels  every  3  mo  if 
stable 

6.  Prevention  of  recurring  mania  and 
depression 


94 


Study  Topic:    Chemistry  Screen  Results 
Perceived  Problem:    Inadequate  follow  up  for  abnormal  chem.  screen  results 

Potassium 
below  3.5  or 


Test 

Above  5.5  MG 

Bilirubin 

Creatinine 

Calcium 

Follow  up 

-  repeat  test 

-  repeat  test 

-  repeat  test 

-  repeat  test 

procedures 

-  reference  to 

-  GPT 

-  Creatinine 

-  notation  that 

required  w/in 

drugs  being 

-  notation  in 

clearance 

the  test  was 

one  month 

used 

progress 

-  notation  that 

abnormal  or 

(at  least 

-  notation  that 

notes  of 

test  was 

treatment 

one) 

test  was 

abnormal ity 

abnormal  or 

abnormal  or 

or  treatment 

treatment 

treatment 

Random  Blood 
Glucose  over 


Uric  Acid 

SGOT 

Protein  above  8 

150  MG 

-  repeat  test 

-  repeat  test 

-  repeat  test 

-  repeat  blood 

-  notation  that 

-  SGPT 

-  Electrophore- 

sugar 

the  test  was 

-  LDH 

sis 

-  Glucose  Toler- 

abnormal or 

-  CPK 

-  notation  of 

ance  test  (GTT) 

treatment 

-  note  of 

abnormal  test 

-  notation  that 

abnormal 

results  or 

patient  was 

results  or 

treatment 

fasting 

treatment 

-  notation  of 
abnormal  test 
results  or 
treatment 
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APPENDIX  B 
OFFICE  PROFILE 
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Participant: 

Other  Physicians: 

Date  and  E valua- 
tors: 

Contact  Person: 


APPENDIX  B 
OFFICE  PROFILE 


In  Office  Tests: 


X-Ray: 


Lab: 


EKG: 


Tests  Sent  Out: 


Medical  Records: 


Filing: 


PA  view  of  chests  filed  in  x-ray  files  with  an  abnormal 
or  normal  note  in  progress  notes. 

CBC,  Sedrate,  Urinalysis,  Blood  Sugar,  Occull  Blood, 
Vaginal  and  Urethral  gram  stains, 
3  and  5  hr.  glucose,  Hemoglobin,  Hematocrit,  Uric 
Acid.    These  test  results  are  recorded  on  a  pink  form 
in  the  record.    This  form  is  used  for  multiple  tests 
and  in  that  manner  works  somewhat  like  a  flow  sheet. 
(Attachment  A) 

An  EKG  and  Bicycle  EKG  are  done  in  the  office  and 
pasted  into  a  special  folder  which  is  inserted  into  1he 
office  record.    Normally,  there  is  no  notation  in  the 
record. 

Multi-chem  screens,  pap  smears  and  other  special 
tests  are  sent  to  Northwest  Medical  Laboratories. 
Results  of  all  tests  except  multi-chems  are  stapled 
in  the  front  of  the  record.    The  chem  screen  results 
are  loosely  filed  in  the  record  with  those  associated 
with  the  latest  visits  in  the  front. 


Records  arc  filed  by  number.    A  rolodcx  containing 
the  patient's  name  and  record  number  is  maintained. 
The  rolodex  is  ordered  alphabetically  by  name.  The 
record  number  will  be  used  for  the  patient's  number 
on  the  abstract  form. 
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Forms:  Laboratory  tests  done  in  house  are  recorded  on 

a  pink  form  and  laboratory  tests  sent  out  are 
recorded  on  the  appropriate  report  form  from 
Northwest  Medical  Laboratories. 

Progress  notes  are  handwritten  on  notebook 

paper  with  the  date.    If  it  is  a  physical  exam, 
physical  exam  or  periodical  examination  is 
typed  onto  the  notebook  paper  at  the  beginning 
of  that  visit.    Initially,  notes  are  somewhat 
difficult  to  read,  but  appear  to  be  concise  enough 
to  facilitate  interpretation. 

X-ray  and  EKG  reports  are  recorded  as  indicated 
above.    Telephone  conversations  are  not  regularly 
recorded,  unless  it  relates  to  a  drug  refill.  There 
is  a  green  drug  refill  sheet  where  all  refills  are 
recorded. 


Order  of 

Notes:  Physician  #      uses  a  traditional  record  format. 

Notations:     There  appear  to  be  no  special  notations  of  relevance. 


Billing  Records:        Since  physician  j     hacl  already  indicated  that  the 

gonorrhea     criteria  would  not  be  used  in  his  office, 
and  there  were  methods  for  identifying  patients  for 
the  pelvic  exam  and  hypertension  criteria,  we  did 
not  review  his  billing  records. 


Appointment  Re- 
gistry: The  appointment  ledger  indicated  the  name,  the  type 

of  visit  (the  amount  of  time)  and  the  problem  if  one. 

Physical  exams  can  be  easily  identified  from  the 

appointment  ledger. 
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General : 


Clinic 
Hours: 

Working 
Space: 

Uniforms: 

.Approach: 

Hypcrtcn- 
sion : 


Pelvic: 


]{ati  lication: 


Comments: 


M.FMC 
5/9/77 


9-5,  Monday  through  Friday.   

usually  takes  Thursday  afternoons  off. 

Upstairs  conference  room. 
Not  normally  worn. 

The  approach  to  locating  charts  will  be  as 
follows: 

Physician  #  .has  kept  logs  of  his  hypertension 

patients  and  these  patients  will  be  reviewed.  He 
now  has  about  45  patients  on  his  log  and  we  expect 
by  the  time  we  can  schedule  review  for  his  office, 
there  will  be  50. 

These  will  be  located  by  pulling  charts  for  females 
who  made  appointments  for  physicals  as  noted  in 
the  appointment  ledger. 

Physician  #      indicated  that  he  would  ratify  the 
hypertension  criteria  as  it  stands,  he  would  ratify 
the  pelvic  criteria  as  it  stands  except  for  the  re- 
quirement for  a  pap  when  the  cervix  has  been  re- 
moved for  non-malignant  reasons,  and  that  he 
would  not  ratify  the  criteria  concerned  with  gonorrhea. 
Consequently,  we  will  not  use  the  gonorrhea  topic  in 
his  office. 

Physician  #     indicated  that  he  may  be  able  to  find 
someone  in  his  office  to  do  some  reabstracting, 

perhaps  a  month  from  now  after  (  )  is  married 

and  settles  down.    He  objected  to  the  rate  differences 
for  reimbursement  between  HNs,  LPNs  and  medical 
office  assistants,  because  he  felt  that  well  trained 
nvadical  assistants  could  do  an  optimal  job  in  the 
office  and  an  RN  was  not  necessary. 
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APPENDIX  C 
VARIATIONS  BY  PHYSICIAN  AND  TOPIC 


TOO 


- 


- 


VARIATIONS  BY  PHYSICIAN  AND  TOPIC 


APPENDIX  C 


PHYSICIAN  #2 

PELVIC  EXAM:  Three  charts  were  reviewed  for  1976  had  no  rectal 
exam  found  by  the  abstractor    the  exams  were  done 
in  all  three  cases.     These  charts  were  reabstracted 
by  the  Field  Coordinator  with  the  same  results  of 
no  rectal  exams  being  noted.     After  the  follow-up 
review  it  was  found  that  there  was  a  physical  exam 
done  in  1975  with  no  rectal  exam  being  noted    It  is 
possible  that  the  1975  PE  was  abstracted  and  not  the 
1976. 

Another  case  selected  for  review  showed  that  while 
the  abstractor  did  not  find  the  Pelvic  and  Bimanual, 
Rectal  exam  and  Pap,  review  of  the  charts  showed  all 
the  procedures  as  being  done. 

Two  other  cased  reviewed  had  variations  of  no  pap 
smear  being  done,  one  case  the  exam  was  not  done; 
this  may  have  been  due  to  the  patient's  age  (80yrs). 
The  second  case  reviewed  showed  that  there  was  a 
notation  on  a  flow  sheet  stating  the  exam  was  done 
by  another  Physician. 

These  problems  encountered  were  due  to  abstracting  . 
errors,  however,  it  should  be  noted  that  the  charts 
in  this  office  took  a  lot  of  concentration  and  sear- 
ching to  find  the  information  needed. 

PHYSICIAN  #3 

PELVIC  EXAM:  Variations  in  the  criteria  were  not  discussed  at 
this  time,  a  follow-up  review  will  be  at  a  later 
date . 

PHYSICAIN  #4 

PELVIC  EXAM:  Variation  in  the  criteria  were  reviewed  in  two  cases, 
1)  pelvic  exam  and  pap  smear  were  not  done  because 
the  PE  was  done  for  a  specific  diagnosis  (Paresthesia) 
The  physician  did  agree  however  that  the  patient 
should  have  a  pap  smear,  2)  PE  was  done  without  a  pap 
smear  or  pelvic  exam,  no  reason  could  be  found  as 
to  why  the  exam  was  deferred,  however,  It  was  done 
three  months  later  (almost  two  years 'between  exams). 


PHYSICIAN  #7 
PELVIC  EXAM: 


The  three  charts  reviewed  showed  that  while  in  the 
appointment  book  it  may  have  stated  the  patient  was 
coming  in  for  a  Physicail  Exam,  the  patient  was  act- 
ually coming  in  for  a  problem.     No  pap  smears  were 
performed  at  that  time. 
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"VARIATIONS  BY  PHYSICIAN  AND  TOPIC 


PHYSICIAN  tf5 

PEDIATRIC  GASTROENTERITIS: 


In  the  review  of  charts  with  the 
physician,  a  problem  was  found  with 
the  coding  used  in  the  billing  sys- 
tem.    This  type  of  coding  caused  pat- 
ients to  be  grouped  in  this  category  . 
which  were  not  diagnosed  as  having 
Pediatric  Gastroenteritis. 

Also  a  problem  was  charting  conven- 
tions not  being  known  by  the  abstract- 
or before  abstracting  began. 

Three  charts  reviewed  showed  clear 
liquids  noted  as  "taking  fluids", 
or  "H2O",  and  NPO  noted  as  "diet". 
Two  other  charts  reviewed  showed  a 
notation  of  "standard  diet"  used  also 
for  NPO  and  clear  liquids.     These  notes 
resulted  in  abstractor  errors. 

One  chart  reviewed  Indicated  an  abstract- 
ing error  for  finding  no  notation  for 
"well  hydrated  child". 

The  CBC  criteria  was  questioned  by  the 
physician  as  revelan.t  criteria.  Be- 
cause of  the  overall  low  rate  of  com- 
pliance for  the  CBC  criteria  by  all  the 
physicians  it  should  be  noted  as  a  ques- 
tionable criterion. 


PHYSICIAN  #20-24 
PEDIATRIC  GASTROENTERITIS 


Variations  in  the  criteria,  in  review- 
ing the  abstracts  some  of  the  patients 
in  the   study  did  not  have  a  history  of 
vomiting  and/or  diarrhea.     This  was 
found  to  be  a  problem  ftith  the  physician' 
coding  method.     The  physicians  stated, 
"We  use  the  diagnosis  of  gastroenteritis 
to  include  many  cased  which  would  be 
more  properly  just  gastritis  or  enter- 
itis, therby  negating  the  chance  of  us- 
ing the  "NPO"  designation.     Many  of  the 
cases  were  seen  after  the  acute  episodes 
of  vomiting.     We  were  primarily  dealing 
with  diarrhea  although  the  patient  would 
be  coded  out  as  gastroenteritis". 
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